Simulations, Logs & Dummies


Meters, Warcraft Logs & Statistics: Part 3

A continuation of the blog series I started recently (Part 1, Part 2).

Another common question raised by players is the “My sims say I should do X DPS but I only get Y DPS on dummies or from WCL parses” one. Unfortunately all three approaches can’t reliably be cross-compared with each other for specific details.

Simulations

Simulations are only ever as good as the system that generates it and the action list used, so there could potentially be differences between in-game results and simulations. Sims also run in a “perfect” performance manner, although SimulationCraft does have some skill settings that you can tweak to lower this. This means that even though you may be performing your rotation perfectly in game the simulation results will simply be better because it can evaluate situations & make literally split second decisions that human players simply cannot match.

The fight type used in sims are also important. Both SimulationCraft and AskMrRobot have various fight options/boss scripts that can be used to approximate an in-game encounters, but using these to properly optimise your setup will require a lot of different encounter simulations to either have a different setup for each encounter or a merged general one for all encounters in an instance. Most players will use the standard “Patchwerk” model to optimise stationary single target damage, as this comprises the majority of DPS check situations. Optimizations for single target are also more significant than multi-target, purely because the damage output against multiple targets is significantly higher and a much shorter duration.

Simulations are primarily used for optimizing rotations, talent choices, etc in an ideal environment where we can reproduce results fairly easily. Repeating results in-game will require a lot of time in addition to similar random event behaviour, usually where you manage to avoid being targeted with a random boss ability.

Testing for Dummies

Training dummies are useful for confirming the behaviour of abilities in-game, as you can go in & out of combat at will and don’t have any mechanics to worry about. The issue with citing dummy results is a combination of that lack of encounter mechanics plus the subjective nature of individual dummy tests. One player could give remarkably different results between different dummy tests depending on the test length & cooldowns used. Stacking Bloodlust with lots of cooldowns and reporting on 1 minute will show a higher result than doing the same thing for 6 minutes.

In-game testing like this also has that PEBKAC weakness: Problem Exists Between Keyboard And Chair. You won’t be able to perfectly reproduce a rotation from simulations so this will skew your results somewhat, as will lucky (or unlucky) proc streaks. Simulations rely on thousands of iterations to return an average value and avoid this issue

Encountering Bosses

Actual log results on WarcraftLogs or in-game damage meters will be different again, and usually from each other too. The mental overhead required for boss encounters means that you’ll be concentrating on that mechanic that would kill you or cause a wipe than perfectly executing your rotation. Even players with amazing skill levels will see a performance decrease between what they can do vs a training dummy and in a boss fight.

Note that I said that damage meters can be different from each other. This depends on a few things like when they start/stop including data. For example, Skada starts charting damage when you start attacking, whereas Details goes off when you enter combat. Timing issues like these will result in differences, since Damage Per Second is based around time.

Boss fight results are also affected more by events out of your control, either from encounter mechanics to how much DPS the rest of your group can pump out. The shorter the fight the more effects like Bloodlust can increase your DPS, especially when stacking multiple cooldowns.

Comparisons

You can only make approximate comparisons between all three data sources, and unfortunately as the ability to reproduce results decreases the more variables need to be taken into account.

Advertisements

About binkenstein
I'm a geek, living in Christchurch, New Zealand.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: