Experimental Pooling Branch

TrinityCore Development TrinityCore Code Discussion
1

Hi all,

I’ve been making (and still am) some changes to the pooling in TC (3.3.5 branch, but I don’t know - it could be extended perhaps).

Essentially one of the things I’ve noticed is that in certain zones quest mobs were a real problem. Not enough spawns of them. Now, aside from more spawns being made in the DB. I also noticed a feature that is on retail (and I’ve seen it a few times, specifically when in starting zones on a new realm for example).

That is, if many people are killing a certain class of creature - then that creature will start to respawn faster. Such that people can still finish their quests. Now, I don’t actually think they respawned faster, I think what they actually have is a minimum quota on their pooling system. Such that when the number of creatures in that class drop below that limit it will start to clean them up and respawn in an expedited fashion.

The solution I’ve made is a bit crude. It doesn’t really have a delay in place. But, probably it will work almost the same in practice.

I think this could also be extended to resource nodes. Such that (only normal nodes) would have a minimum limit. What I need to add here though is something a little different. But when I move onto that area of things I’ll describe why and what I mean.

For now, here’s the paste from the readme about the progress so far, and a link to the repo should anyone want to try. I’d like some feedback from experienced core developers on the changes (I program for a living, but not with c++ so input would be appreciated), and maybe some people that can actually try and see what they think.

Disclaimer, by default the database really has zero use of pooling for creatures. So, you will need to make some useful data before you will notice any difference at all!

https://github.com/pete318/TrinityCore/tree/pooling_wip

The aim of this branch is to update the pooling system as follows:

  • Create a possibility to specify a minimum number of spawned objects
  • Ensure the maximum value are actually spawned
  • Ensure that the minumum number are maintained in the world

Completion: 40%

  • Databases and Pool functionality updated to handle minimum value in template: 100%
  • Added pool information to npc info (creature pool): 100%
  • Spawn insurance of 100% max spawn if possible: 80% (this seems to work, but I have seen npc info report not all spawned. So need to check)

Creature specific:

  • When creatures in pool drop below minimum, then process is as follows: (100% but needs testing)
  • looks for a dead despawned creature in the pool, if found - instant respawn
  • if none despawned, looks for a looted corpse. If found, despawns corpse
  • if no looted corpses found, then finds an unlooted corpse and despawns it
  • Creature class upon creature despawn, will check if an expedited respawn is required. If so, will respawn right away.

Resource Node (maybe all game objects in pools) specific:

  • When nodes in pool drop below minimum, then process to be determined to maintain minimum nodes in pool (0%)

To Do:

  • Currently more is exposed in PoolManager class than needs to be (this is due to extra detail being shows in npc info for debug purposes during development)
  • I’m not too happy about making the ExplicitlyChanced and EqualChanced lists directly available from PoolGroup. But, it was the fastest way to achieve this.
    It doesn’t seem to pose a terrible problem.
2

Dynamic pooling (I think that’s the name that this thing is called) is one of the oldest features that we never had so thanks a lot for giving it a shot.

The easiest way to get feedback is probably by opening a Pull Request from your branch to our 3.3.5 branch (here). Just make sure to write “[WIP]” on the title of the PR so we know that it isn’t ready to be merged.

3

Thanks, I was reluctant to do so because I believed the PR system was for finished work.

4

Blizz actually never “respawns” creatures the way we do it (they just spawn a new one instead of reusing dead ones) - but that is something way beyond pooling system and we are not expecting anyone to implement it (the reason blizz does it is to not fuck anyone with unlooted items)

5

The pooling system actually does do that, and did already.

It surprised me because at first look, it seemed like it just spawned a quota, then reused those over and over. It actually doesn’t.

How I tested was go to belf start zone and made a lot more mana wyrms. Then I set them to roam like all the others, and then added all guids into a pool with a template to max 20 and min 19.

Also I put some debug messages in the various stages of respawn. When a new creature is spawned (corpse remove and respawn timer up) it despawns the old one and spawns a new random one from the list.

What was a problem (and still is, because I don’t like my solution) is that it didn’t fix any collisions during random selection in the initial spawn. e.g. spawning 60 out of say 100 GUIDs. You will get some collisions (the random number is an already spawned item). As standard it doesn’t try and fix this. The dirty change I made was that if there are enough spawns to make up the maximum required, it will call itself until that number is reached.

What I might need to fix is whether the respawn part also takes this into account. Otherwise the number of mobs will slowly dwindle on a curve as the respawns collide with existing spawns.

You could actually do it fully the blizz way, in theory with not a horrible amount of change actually. Something to look into long-term. What I was aiming for was a more blizz-like feel.

EDIT:

Actually now you’ve mentioned it. I think it’s probably worthwhile changing it to work that way. Since this disappearing corpse syndrome (which really wouldn’t happen with a decent sized pool, but - still it can be avoided) bugs me a little.

So, how it will in fact work. Upon creature death, the “Handle creature death” will be called as normal. However, it’ll work somewhat different. All of this only happens if the creature is in a pool. If not, normal action applies.

In this case, if there is a shortage of the creatures in the pool, then an expedited spawn will take place. But, in this case it will just pick one from the pool that isn’t active and spawn it. If there isn’t a shortage, then the respawn timer will be set as normal.

When the creature’s corpse would usually be removed. It’s usually invisibly teleported. Insteas the creature will be entirely despawned and removed.

When the respawn timer is up, a creature will be spawned out of the pool.

This has the bonus in that creature corpse time could be extended when not looted at least without changing the spawn time.

The downside is that there needs to be substantially more creature spawns in the database than the amount you want spawned at any one time, to allow for unlooted corpses lying around.

The TRULY blizzlike way would be to have the template and use the creature table guid as only an identifier for pooling and spawn point. Generating the guid on the fly.

Honestly, the last way would I think be how they genuinely do it (hence how you can have a creature spawn on top of you just after you kill it, with the corpse still on the ground). But, would require the most work…

Making me think now…

EDIT again:

I actually had another look at the chance code (running iterations feels dirty and probably will cause adverse load if pools are actually used everywhere as they should be). I noticed something that will be breaking and I’ll need to fix.

Currently a % chance to spawn a specific template entry only works if there is only 1 maximum to spawn. Which if we wanted to say spawn a field in barrens, where there’s maybe 3-4 kinds of creature. Most with same/similar spawn chance but then a few with a much lower (the raptors)… Then it won’t work.

I can see why it was done this way. Using the pool resources available it’s actually quite hard to make this chance happen directly. It might be possible if the spawn list was made in SQL. Otherwise it gets a bit more complex.

I’ll have a think about this one.

6

So, after a bit more of a think about it. I think the way to do it that will resolve the issue with explicit chance spawns and also make the whole thing “more blizzlike” is to actually not use the creature table for pooled creature spawns. Hear me out.

pool_creature will be extended to contain the spawn information contained in a normal creature record. However the guid will point to a creature template rather than creature. There must be only one record for each kind of creature per pool. The chance total for all records must either be 0 or add up to 100.

Next, a new table called pool_spawns will be added. It’ll contain the pool_id, a unique id and x/y/z/o (and maybe some more, not sure yet).

When spawning a single object, first a creature will be chosen in a weighted fashion based on the chance total. There’s many ways of doing this and it’s much less of a headache than trying to work with the pool contents the way it’s done now.

Next, a spawn point will be chosen at random. The random creature will be spawned at the random spawn point and the point marked as “in use” (it won’t be considered for use).

When a creature dies. The respawn timer will start right away. With a new spawn chosen in the same way.

Now, dead creatures will work in a different way. As soon as the creature dies, the spawn point will be released and available to re-use. Once the corpse decay time is up, the creature will be despawned and removed from the world.

I think in this way, we get the random spawns in a blizzlike way, which will support a weighted spawn chance for multiples of creatures. Plus, it will also support the possibility of respawn times lower than corpse decay, and the ability to extend corpse decay without hurting respawn times. However, ONLY if all creatures are put into pools (quite a task I’d say).

For quest creatures it could work well. The named creature goes into a pool on it’s own with a short respawn time long corpse decay time. That way he can be killed by a player and it won’t be overly long before the next player has to wait for the respawn.

My only worry would be the effect on performance of always spawning and despawning creatures in this way. Also, there might need to be some kind of guid re-use system produced, or a global change to uint64 for guids. Since with virtually every creature suddenly becoming spawn on demand, the use of guids will increase greatly.

Anyway, some comments… Before I embark on this quest, would be appreciated.

7

There was a discussion about this some time ago (can’t find it right now),

and i think we talked about how blizzard does not have the “creature” table, but instead a list of possible spawn points, so you are on the right track /emoticons/default_smile.png

8

Well, I don’t think we can just “drop” the creature table (yet)… I was thinking more of a transition. Where creatures in pools use the creature_template, an extended pool_creature (with some fields from creature table) and a pool_spawnpoints table to define the possible spots.

Of course, if it’s possible (and it should be) to put all creatures inside the pooling system… The creature table could be dropped entirely I think.

Alas, the code might not be the hardest part… Putting the creatures into pools, and converting to spawn points all those in them will be the time consuming part.

9

Thanks for this but I don’t think despawning unlooted creatures simply because pool count is low is a good idea.

I commented on your PR since I don’t know where the conversation will actually take place. In a nutshell, players may be surprised when they go back to loot and skin Onyxia or The Beast that they’ve been despawned prior to the expected unlooted despawn defined in the CONF.

10

De spawning them is blizzlike, I guess. Up until a very late Hotfix, even rates sometimes de spawned before being able to loot as max corpse/unit count was too low. Not good, but some limiting is needed.

11

I think that only applies to heavily populated zones and NPCs that are quest related. It was Blizzard’s way of trying to streamline questing so you didn’t have to wait 15 minutes after someone came in and AOE killed everything. Now they use shared tagging so anyone can get quest drops from a kill as long as they help, grouped or not.

Anyway, if Trinity decides to go this route it should be optional. There are many other customizable parts of the core and people should be able to continue to use a set respawn timer.

12

I don’t remember looting part, but as for killing for some mobs you get kill credit even killed by another person and you aid and for some no, maybe there is some flag somewhere or it varies depending on mob (normal vs elite)

13

This is the thing. I don’t actually think they change the rates. That would take more processing time. I’m fairly sure that they have a minimum number in a pool that must be upkept. That way it’s more event driven. When a creature dies, you just check the pool alive count and expedite if it drops below a certain value.

Right now in the experimental version I respawn them right away. I think Blizzard do a batch respawn every minute or so though.

Just to clarify. The overall plan isn’t to despawn the existing corpses. This is just to work with the current way it works. The ultimate plan is to switch pooling to be designed differently. e.g.:

Pool ID -1:N-> Creature_ID (from template) -1:N-> Spawn_Point

Spawn points not being tied to the creature. They’re tied to the pool as a whole. Creatures then being dynamically spawned from the pool (using a better weighted random chance system because we wouldn’t be considering the whole pool, just the creature type) into a random spawn point (that doesn’t already have an active, alive creature attached to it).

My main issue here is whether the guid is cached, or the creature ID (from template). If it’s the ID from template. It’s fine. But if the guid is somehow cached… Then we might have a problem. My gut feeling (I’ve not looked into it yet) is that the guid is only used by TC to track the objects internally. But, it needs to be checked. If it is indeed fine to spawn dynamically like this. Then, I think it’s the way to go. Because then we can use more blizzlike values. From what I’ve read and tested myself the values (well at least as-of 3.3.5) were something like this:

Creature dies, and is looted. Corpse will be despawned after 2 minutes.

If unlooted, it will remain for 5 minutes after death.

Checking the loot (right click) but not taking any (or at least leaving one item) resets this 5 minute count.

In theory someone with enough time on their hands could keep an unlooted corpse there forever. But, I doubt that would be a problem.

The above is for normal creatures. Ultimately my thinking is that there should be a default variable in the config file, as well as the ability to set custom values per pool or even creature item per pool (but not per spawn point, that’s just silly). My feeling is per pool is good enough. Unless you have rares somehow in a normal pool and want these to have a longer corpse decay time… Then it’s a different story.

Because they’re then dynamically spawned. The corpse doesn’t become linked with the respawn. So the respawn timer can start on creature death. The replacement creature can even respawn while the previous corpse is still on the ground. There’s questions as to whether there needs to be zone limits for corpses and some kind of expedited despawn if it gets out of hand… I’m not sure.

Anyway as I say the longer term design I’m making is trying to cover all eventualities (and ultimately could be rolled out to gameobjects too. Which could help stop the double node spawns for example). I’m thinking that for example you would have a mining pool entry for Saronite. It has all the node points. With Saronite at 99% and Titanium 1%. With a minimum and maximum just like for creatures.

As I mentioned on the PR. I’m somewhat busy with other things. But, in my spare time writing this up is priority. So, probably for creatures at least I’ll have something I think people can review here early in 2015.

Of course, the hard part is getting creatures into pools. So, for that reason there need to be some management tools added. For example targetting a creature, adding all creatures of that type within “x” radius also to the pool. Temporarily spawn all spawn points (to see where there are dead zones to add new spots). Some way to highlight creatures that are part of a pool. So catch missing ones. etc etc.

So, just to be clear. The long term aim here is something much more substantial. But, it’s something that I want to design rather than build first. Get input and then maybe deploy.

14

Checking the loot (right click) but not taking any (or at least leaving one item) resets this 5 minute count.

This I think is a bad idea. A player could kill a rare elite and if they wanted to be an ass, constantly reset the timer so it won’t ever respawn.

15

I think you’re not paying attention to the other changes I also stated would come in at the same time… It wouldn’t matter. The creature would respawn while his unlooted corpse was still at his feet. Just as it would on a real server.

If you’re still subbed, you can even try it. I believe (although it’s been a while since I checked) it still works this way.

16

You’re right, I did gloss over that part. In that case, this doesn’t sound like a bad idea.

On the other hand, my memory from retail is that rare elites took hours to respawn because they’re well, rare.

17

That’s all to do with the configuration. At the moment my thinking is that there is a default value in the configuration for corpse despawn non looted, corpse despawn looted and respawn (already there). Except now there won’t be despawn+respawn. Only respawn time will matter for that part.

Also in the creature table for the pooling side there will be values for all 3 too. So it can be set creature per creature. If 0, it uses config otherwise the value from DB.

This also allows the rare, but possible “instant respawn” mobs. For example those around the cannon quest for DKs. I currently did this using creature AI C++. But, we shouldn’t have to resort to this.

As I say though. The current commit was a quick change that gave something a little more like what I’d see on a real server, while not totally re-designing the processes. For pooled creatures at least I think this fundamental change would be needed.

At any rate I’ll post up the finished design here for some review. Since, it’ll be quite some work to change and I’d not want to do it for nothing. So ideally there should be general agreement that my plan points in the right direction.

18

After some time, I’ve finally finished (well to an extent) the creature side of this:

https://docs.google.com/document/d/1E1GVl5DL4_IYi6aQsgVnqhu0Cy7mAF5pYj4Er6zZvqw/edit?usp=sharing

There is one important note. Loading the server back from a saved state regarding loaded/unloaded grids. While I believe it’s likely possible to get the grids loaded back correctly, albeit with some new state tables needed to do the job. I have to wonder whether it’s a needed feature. I have no solid evidence of this, but I’m fairly sure when real servers were restarted spawns were reset. So, I wonder if this should be the case for TC too. In any case, it’s a work in progress area.

I’d like some feedback especially if I’ve suggested something that isn’t possible or just plain isn’t correct.

Thanks.

19

Everything looks good, however, I’m not sure about the respawning part. Once a creature dies, its spawnpoint is added to inactive spawns, which means that it can appear in the same spawnpoint where it was killed. I haven’t seen such thing happening in blizz so I think they might be using FIFO container for spawns. I.e. last killed spawnpoint will be last to respawn. I might be wrong tho

I will also take a look at grid stuff later because it seems over complicated. And I think we can not compare it to blizz, because they probably has all grids loaded all the time which simplifies a lot of stuff. Unloading is useful only in low pop servers which run on mediocre hardware.

20

I never saw it before Cata. I actually think prior to cata, they had many more spawn points than creatures. But, in cata when they redesigned the old zones I saw many times in busy zones it was possible to kill a creature and have a new one appear right on the same spot.

I don’t think we could do it on corpse removal. Because then we hit the same problem we have now. Where, if many many corpses remain around unlooted it can mess up the respawn process. If we put it on a timer, it’s fine but adds an extra level of complexity.

FIFO might make sense, but there needs to be some randomness to it. What could be done is have semi FIFO. So for example, there’s 50 spawn points. Creature dies, we add the spawn point back to the bottom and we choose one at random from 1-40 (meaning the last 10 dead creature positions won’t be considered). That probably wouldn’t take a huge amount of work to accomplish. But, probably a configurable percentage that can be tweaked with testing could be used to decide how many of a pool shouldn’t be reused.

For the grid stuff, there I don’t (yet) have all the understanding of how it works. I started looking at it, realised it’s quite a minefield and wanted some opinions as to whether we want to support the entire current functions, a subset or to say if using pools, you don’t get to unload grids. Or, if we unload grids, we don’t get to save grid spawn state on server restart.