And I'm at it again! Last round against this mechanic (http://forums.worldofwarcraft.com/thread.html?topicId=110522591) ended up without a definitive result, but indicated a one roll system. The issue observed in that experiment was that the margin for error was overtaking the margin between the two expected results, so the results required a whole metric ton of datapoints (10,000 wasn't really enough to prove it, sigh).

Thanks to the wonders of PTR premade characters I had a level 70 rogue available to me today for testing, and did the following:

Conditions:
-Level 70 rogue bare spec wearing no resil gear
-Level 70 mage bare spec wearing no +hit gear, crit rate of 14.84% from gear.
-Enter a duel, cast rank 1 scorches at the rogue with Cloak of Shadows active, record hits crits and resists.


Notes on the test environment:

-I had an addon recording the data for me, watching the rogues buff array for Cloak of Shadows being active, it ignored any hits that occured within .25 seconds of the buff fading to account for latency screwing with it.
-Level 70 vs Level 70 was chosen since I have observed in other testing (http://forums.worldofwarcraft.com/thread.html?topicId=108219860) that crit rates involve target level aswell, and one can assume that those equasions are weighted to have tooltip values at equal level.
-Timetable of spellcasting went as thus:

0.0 - Start casting Scorch on mage
0.5 - (ish) activate Cloak of Shadows
1.5 - Finish first scorch, start second
3.0 - Finish second scorch, start third
4.5 - Finish third scorch.
5.5 - Cloak of Shadows fades

This assumes no latency, I was usually landing the third scorch at around T+5.0 seconds or so. This means three datapoints a minute at an absolute fastest (oyvey)



And the results!


Datapoints taken: 200
Resists: 186
Hits: 11
Crits: 3

Expected Resist Rate: .04 + .9 = .94
Observed Resist Rate: 186 / 200 = .93
Delta: .94 - .93 = .01


So resists were nicely in line where we expected them to be with this small sampling. Now looking at crit rates. If we expect a one roll system, every single spell that isn't resisted should be a crit, since our 14.84% crit rate goes over the 94% miss rate. If we expect a two roll system, we expect 14.84% of our non-resists to be crits.


Expected Crits by One Roll: 200 * .06 = 12
Expected Crits by Two Roll: 200 * .06 * .1484 = 1.7808
Observed Crits: 3
Delta from One Roll: 12 - 3 = 9
Delta from Two Roll: 3 - 1.7808 = 1.2192


The mere presence of non-crit hits in the samplings indicates a two roll system rather convincingly. Even though it conflicts with the other data I gathered, my other data conflicted with other peoples data, so I'm at this point willing to say two roll looks like the most convincing model.

Any commentary from the peanut gallery?

12 is not a reliable sample size...

We already knew it was a 2-roll system - Though I do admire your effort to actually solve confusion in WoW rather than just post questions in the forum and determine answers by Who Yells The Loudest.

If you want to do more testing, here's an alternative method...
Get a 60 mage with combustion and/or shatter and fight a lvl 66 mob. (Best to have a 70 tank it)
A lvl 60 player has a 50% chance to hit a lvl 66 mob - with a few stacks of combustion or a frosbite proc/nova, the players crit rate is 50% or higher. On a one-roll system, a normal hit simply would not fit on the hit chart (50% miss, 50% crit, 0% hit) - ANY non-crit hit would be evidence of a 2-roll system.

No dice on the 60 vs 66, we cant accurately predict crit rates with level disparity, I showed that with the Ice Lance on the PTR test when I observed a 7% inflation vs lower level targets. Any test that relies on level disparity to drive up the resist rate will fall victim to the crit rate also becoming skewed in ways that we cant predict accurately and thus are too much of a variable.

---

Q u o t e:
No dice on the 60 vs 66, we cant accurately predict crit rates with level disparity, I showed that with the Ice Lance on the PTR test when I observed a 7% inflation vs lower level targets. Any test that relies on level disparity to drive up the resist rate will fall victim to the crit rate also becoming skewed in ways that we cant predict accurately and thus are too much of a variable.

---

Higher levels does not affect crit rate... a fight against any single boss in the game is proof of that.

What you're making is an assumption from evidence outside a controlled environment.

The only true way to test that is to take a level X character, and a level X-3 character that is hit capped, and cast a few thousand spells to get a good reading of their crit rating. Or, if you cant consider PvP a viable test bed, a hunter kiting a target three levels higher than him, somehow capped in hit gear, and using such a trashy weapon that it wont kill the target within a reasonable sampling of data. But that will only tell you white damage (unless you only count yellow hits, which means 1/8 second arcane shot samplings, effectively).

And what you're promoting is a thought that theres a mechanic that hard-caps at equal levels, which I find unlikely given the way other mechanics work in the game.

Hmm, the above sounds nasty.

What I'm trying to say, is that you don't have nearly enough control over a given boss encounter to definitively research anything in it. There are far too many variables involved, and your sample sizes are way too small.

How many spells do you cast on average in a boss fight? if it lasts 5 minutes of fireball spamming, thats 5 * 60 / 3 = 100 casts assuming no lag, no other actions that trigger the GCD, etc.

If your crit rate varies from 30% to 29% against level 73 targets, you have absolutely no way to observe that in those conditions without doing that boss fight another 50 times or so, and over those 50 times your conditions will change so much that the data can't be reliably combined together to form anything meaningful.

What I'm doing with my research is actual research, such chaotic uncontrolled conditions do not return meaningful data.

We've got over 2 years of data against boss mobs to come to the conclusion that your paper-doll crit rate is a reliable predictor of boss fight crit%. There is a near endless supply of data that indicates a mob being higher level has no impact on crit%.

Continue researching all you want, the data out there is plenty meaningful already. Again, I'm not really sure who you're doing the research for - anyone thats looked into the matter can see its a 2-roll system. It's like trying to prove evolution - anyone that you would want to convince is working off a different set of information to begin with.

If it were a 1-roll system, those that have a 250% +dmg bonus to crit would possibly be better off with a lower +hit rating and higher +crit.

Think about it, if I could gurantee a crit 100% of the times I hit, and only hit maybe 60% of the time, it would possibly be better than a 20% crit 90% of the time.

(just made those numbers up, but you get the idea of being able to mess with the stats to get a possible better outcome of almost-always critting and just not hitting as much )

The variance vs level 73 targets is likely extremely small. The inflation is around 7% on level 1 and 2 targets at level 70, assuming its a linear pattern (which isnt a terrible assumption, but isn't one that can be used to accurately model anything), that would lead to .1% crit difference per level difference, so a level 73 target would have a .3% lower observed crit rate than a level 70. Considering any sample size against a level 73 (or 63 back in the day) is extremely limited in any way that can be used for actual scientific proof, I'm not surprised not many people think crit deflates vs higher targets.

What you're really missing is the scientific method, and what the purpose of my research is. If I wanted to know what the masses think I'd hire Irontygress and Lhivera as consultants and have them answer all my questions. However, comma, the masses have been *WRONG* on occasion.

I showed this with Imp/Emp Frostbolt, I showed this with Pyroblast. Both were inaccurately represented by most people (Imp/Emp with how they were handled when both were present, Pyroblasts coefficient in general was wrong) to include the experts.

Controlled testing returns true results and proves mechanics. While the years worth of data gathered from data mining addons turned on and raided with for months do generate good indicators of mechanics, to fine tune those to proven mechanics with definitive behaviors requires controlled testing.

The reason I am working on One/Two roll is that every test I have seen has relied on the level disparity between caster and target to reduce the table of hits/crits. The Ice Lance test I did showed that target level matters in the crit equation (Corroborated by Irontygress in the thread http://forums.worldofwarcraft.com/thread.html?topicId=108219860), the mere indication of that calls into question the results of those tests enough for at least some review.

Using your example of a 60 vs 66 target and using shatter / combustion to drive up crit rate. Having a 50% miss rate is nice, but you're using level disparity to drive that miss rate up, which will make your crit rate different. Maybe only .6% different, maybe much more, that mechanic is very difficult to explore accurately without first establishing one/two roll definitively. What this test with CloS does is the same basic thing, pushes hits off the table of a one roll system by generating a 94% miss rate on the rogue without using the level disparity to do so, and thus preserving your crit rate.

---

Q u o t e:
If it were a 1-roll system, those that have a 250% +dmg bonus to crit would possibly be better off with a lower +hit rating and higher +crit.

Think about it, if I could gurantee a crit 100% of the times I hit, and only hit maybe 60% of the time, it would possibly be better than a 20% crit 90% of the time.

(just made those numbers up, but you get the idea of being able to mess with the stats to get a possible better outcome of almost-always critting and just not hitting as much )

---

While that scenario is mathematically possible... I'd say its just about impossible to generate, and if you did generate it, it would have to be with level disparity (since I know of nothing that reduces your chance to hit by that much), and then you'd be extreme low level and your damage output numbers wouldn't be on par with the hit points of the target.

Fine then, go look back at the set of data in your first post - the fact that you hit there when a one-roll system wouldnt allow it should be proof enough for ya.

But just some quick thoughts -
-Ice lance test only proves a relation between lower level mobs and crits, nothing regarding higher leve mobs.
-Your sample size is too low -- your results fall in the margin of error for either set of assumptions
- It is already proven and confirmed by Blizz that PvP hit mechanics are different than PvE, so you have no verifiable reason to believe what you test against a Rogue has any correlation to PvE.

Test away -- If I wanna know how my spells behave against higher level boss mobs, I'm gonna make my conclusions off the millions and millions of casts against higher level boss mobs rather than a couple thousand against a 70 rogue.

---

Q u o t e:
Fine then, go look back at the set of data in your first post - the fact that you hit there when a one-roll system wouldnt allow it should be proof enough for ya.

---

Isn't that what I said in my first post?

---

Q u o t e:
-Ice lance test only proves a relation between lower level mobs and crits, nothing regarding higher leve mobs.

---

And since the tohit relationship is based off of 96% chance to hit at equal levels, where your hit rate decays vs higher level targets, and inflates vs lower level targets, it's perfectly reasonable to take that as indication that its likely that crits follow the same behavior.

---

Q u o t e:
-Your sample size is too low -- your results fall in the margin of error for either set of assumptions

---

? For this test? or the first round of testing I did on this? On this test any mixing of hits and crits indicates a two roll system, I fail to understand where the lack of samples is a problem here?

---

Q u o t e:
- It is already proven and confirmed by Blizz that PvP hit mechanics are different than PvE, so you have no verifiable reason to believe what you test against a Rogue has any correlation to PvE.

---

I admit this point is worrysome. However I showed in the first round of testing this mechanic that the basic spell behavior at equal levels falls in line generally with the idea of how we think things work, even if I didn't have enough data to prove anything with that set due to the lack of spread of expected values. So I can take that as enough indication to make using the duel as an acceptable test bed.

The big variable is if Cloak of Shadows does the same thing as the level disparity of a level 73 target as far as hit goes. Which is why this two roll proof is flagged with Cloak of Shadows as the tohit difference creator. The next step would be to establish a known actual cap for +hit (is 99% a hard cap or not, basically, an easy but long test to do, a few thousand castings with hit rating way over the cap) against an equal level target, then do the same on a X+3 level target, see if the resist rate is the same, if so, observe the crit rate on the second test and compare it to the first. To do this convincingly I'll need a level 67 mage with over the hit cap gear, arcane is the obvious choice for this (10% hit from talents), but I dont have a 67 mage at the moment, so I'll get back to that eventually once I've done other mechanics.

---

Q u o t e:
And since the tohit relationship is based off of 96% chance to hit at equal levels, where your hit rate decays vs higher level targets, and inflates vs lower level targets, it's perfectly reasonable to take that as indication that its likely that crits follow the same behavior.

---

But its not linear. And its not reasonable to assume it because all the numbers you have deemed 'unreliable' indicate otherwise.

---

Q u o t e:
? For this test? or the first round of testing I did on this? On this test any mixing of hits and crits indicates a two roll system, I fail to understand where the lack of samples is a problem here?

---

No one is arguing the resist rate- you have plenty of data on that... But when you're trying to compare hits to crits, you presented a whopping 14 data points.

I totally agree with you that its a 2-roll system, but your research is no more or less conclusive than the dozens of other people that have looked into the matter. You asked for feedback and the fact is your system is far from foolproof and has noticable flaws. And that's fine because its not exactly a perfect environment to create controlled experiments, but don't declare your work somehow superior or more valid.

---

Q u o t e:
Test away -- If I wanna know how my spells behave against higher level boss mobs, I'm gonna make my conclusions off the millions and millions of casts against higher level boss mobs rather than a couple thousand against a 70 rogue.

---

And again you're missing the point of controlled conditions.


Millions of datapoints are spread over multiple patches.

Millions of datapoints are spread over hundreds of thousands of encounters.

Hundreds of thousands of encounters have different buffs, debuffs, gear, talents, encounter specific mechanics, etc.

Controlled experiments have one set of conditions, and you can control what data come in and record it properly. heres an example for you. Lets say I write a data mining addon that is meant to tell me how often rogues resist me while they have Cloak of Shadows active (since when I first started this experiment I was doing the recording caster side, which proved to be realistically impossible, so its a great example).

You can catch when they cast it initially by an event, and start a counter for five seconds from there. But what if your spell lands just as it comes up, do you know if that was a normal resist, or a CloS generated resist? Do you know if CloS was started a second ago and you just now heard about it due to lag? Do you know when it actually fades? (No, since Detect Magic is canceled by CloS, and you cant reapply during the duration without falling victim to the resists).

The answer to everything there is, you can't tell. The most definitive way to tell is to record the hits target side, since you know when you got the buff, and the duration of the buff from the server (down to 100ths of a second) so you can filter spellcasts that happen too close to the start for lag, and too close to the end for lag.

You, cannot, observe this, caster, side, with any amount of sureness.

The same applies for mass data mining addons, things that are tricky like this require controlled situations and tests to be accurately measured.

I'm really not sure why you're arguing that controlled tests aren't definitive ways to test mechanics, thats how science works, by removing as many variables as you can and doing controlled tests.

---

Q u o t e:
But its not linear. And its not reasonable to assume it because all the numbers you have deemed 'unreliable' indicate otherwise.

---

Source? I'd love to see research that shows the crit rate variance vs level differences.

---

Q u o t e:
No one is arguing the resist rate- you have plenty of data on that... But when you're trying to compare hits to crits, you presented a whopping 14 data points.

---

Really, once I had one hit and one crit I could have stopped the test (which was four non-resists in, three hits, one crit), but decided to continue to see if it fit the model, which it appears to be following, in a very non-convincing manner due to the sample size, but it still is nicely in line where it should be.

---

Q u o t e:
I totally agree with you that its a 2-roll system, but your research is no more or less conclusive than the dozens of other people that have looked into the matter. You asked for feedback and the fact is your system is far from foolproof and has noticable flaws. And that's fine because its not exactly a perfect environment to create controlled experiments, but don't declare your work somehow superior or more valid.

---

Of course it has flaws, all research does, I however view level disparity as a huge variable and have now presented research that agrees with the tests done with level disparity that didn't rely on it, which helps more definitively prove the mechanic.

I'm rather confused by Bronx's insistence that this is not conclusive.

Traditional one-roll theory: normal hits (or crits) should be pushed off the table.

Zaldinar observed both normal hits and crits.

Therefore, the traditional one-roll theory has been invalidated.


Sample size arguments are irrelevant when you have a null hypothesis involving a probability equal to zero. All you need is one cast that doesn't fit the model, and Zaldinar has shown that.

Now then, if someone would like to table a one-roll theory that involves a different kind of pushing off the table, we can test against that.

Not that its not conclusive, just that it was plenty conclusive when this was proven months/years ago.

Muhprid, I think Bronx is arguing the need for the testing more than the testing results itself. I also think he misread several things I posted.

---

Q u o t e:
Not that its not conclusive, just that it was plenty conclusive when this was proven months/years ago.

---

This is effectively peer review and renewed research, since WoW is a living system that changes over time it never hurts to re-prove mehanics. We have seen many times mechanics have been changed without patch notes or releases about them, so while its not likely such a basic part of the casting system would be changed, its possible.

Also, part of the concept of peer reviewing is that new researchers recreate conditions of other tests, and see if they get the same results as an initial researcher. Then they attempt to find different conditions that should (if the model holds) create the same results, which I did. I don't trust level disparity for the reasons I've posted above, and thus chose CloS as my test bed for reviewing, which corroborated the level disparity tests.

---

Q u o t e:
Expected Resist Rate: .04 + .9 = .94
Observed Resist Rate: 186 / 200 = .93
Delta: .94 - .93 = .01


So resists were nicely in line where we expected them to be with this small sampling. Now looking at crit rates. If we expect a one roll system, every single spell that isn't resisted should be a crit, since our 14.84% crit rate goes over the 94% miss rate. If we expect a two roll system, we expect 14.84% of our non-resists to be crits.


Expected Crits by One Roll: 200 * .06 = 12
Expected Crits by Two Roll: 200 * .06 * .1484 = 1.7808
Observed Crits: 3
Delta from One Roll: 12 - 3 = 9
Delta from Two Roll: 3 - 1.7808 = 1.2192

---

If there's confusion, it stems from the fact that the underlined parts seem totally irrelevant.
Really all you would need is Expected Hits by One Roll = 0, Expected Hits by Two Roll ≠ 0. Since actual hits ≠ 0... whammo. And at the sake of being redundant... already proven.