so if i understand this right, and using our example from irc chat…
Lua:
npc = summon(x)
then another script, or cpp, or core unsummons the creature (it will be nill)
npc:moveto(x,y,z) << this will drop a lua error and wont crash core
otherLua command1
otherLua command2
.
.
.
so the command1 and 2 wont be executed becaus it “lua-crashed” on moveto ?
and to avoid id we have to use
if (npc != nil)
npc:moveto
?
I agree that its not a good way.
The way I would prefer to have it just to have unsafe pointers.
If the code accesses them, the code is written wrong anyways and likely crashes. (invalid use of pointers)
I dont see a point in making complicated safety precautions which cause overhead and can be questionable.
Its basically overcomplicating the issue to avoiding crashes, when its basically just all about how you code the script.
One idea was to ditch the lifetime code and think about invalidating all lua objects at the end of a lua function call.
This has not been tested yet, but overall it counters the crashing, at least partly. By partly I mean for example if you unsummon an NPC and then try to use it, its probably not going to work properly. Could be good to code unsummoning for example to invalidate the creature unsummoned.
Anyways. The idea would be that whenever the outermost hook is called (hooks can call hooks ofc), it will invalidate / remove all objects from lua that were used in those calls.
With the addition of coding unsummoning and other functions to invalidate objects they delete, it should be safe.
However, it will disable some functionality as you can then no longer save any object in memory between hook calls.
That is basically correct…Assuming the unsummoning happened between hook / function calls.
But you should maybe state more clearly the situation about the threads.
Like… are you saying that the NPC was unsummoned in the middle of running the function?
Or that the NPC was unsummoned between 2 hook (function) calls?
I’d like to see some examples and how I tie the lua script to a guid. If this is entry based I am less interested.
Anything can be implemented.
To guid, to entry …
As stated sometime before, pretty much anything possible in C++ is possible in lua, since C++ functions can be exposed to lua and the whole system is managed with C and C++.
The way Eluna has it is the same way as Arcemu has it.
There are functions that tie a function to a specific event for a creature entry.
Same can be done with guids.
This could be limited also by the AIName maybe?
However, you already tie scripts through DB to creatures etc.
This can be done as well.
Basically you could use function or table names in database. Lets say you put in DB the script_name lua_hogger_ai;
Then you can have in lua a global table or function called lua_hogger_ai.
The table would contain the functions to be called for each event. (similar to C++ style)
Or the function could tie the script functions to the NPC.
Example1, the way we have it:
-- print parameters on combat, registered for some entry or guid through lua
RegisterCreatureGUIDEvent(GUID, ON_COMBAT, print)
RegisterCreatureEntryEvent(ENTRY, ON_COMBAT, print)
Example2, more like using classes:
-- print parameters on combat, registered for some entry or guid through DB, using table
-- used SQL
-- UPDATE creature_template set scriptname = "some_lua_ai" WHERE entry = xxxxx;
-- UPDATE creature set scriptname = "some_lua_ai" WHERE guid = xxxxx;
some_lua_ai = {}
function some_lua_ai.OnCombat(...)
print(...)
end
Example3, triggering function to do what is needed:
-- print parameters on combat, registered for some entry or guid through DB, using table
-- used SQL
-- UPDATE creature_template set scriptname = "some_lua_ai" WHERE entry = xxxxx;
-- UPDATE creature set scriptname = "some_lua_ai" WHERE guid = xxxxx;
function some_lua_ai(isguid, entryorguid)
if (isguid) then
RegisterCreatureGUIDEvent(GUID, ON_COMBAT, print)
else
RegisterCreatureEntryEvent(ENTRY, ON_COMBAT, print)
end
end
C++ is not used for guids. The assumption is made in C++ that all guids of an entry do the same thing. You can see in db the c++ scripts are assigned by entry.
The regular C++ hooks only allow binding C++ scripts and AI to a creature entry.
But the C++ script itself can be coded to only add the AI to specific guids. In C++ nothing is assumed.
At least I have not yet seen a piece of code where it assumes that all entries have same AI.
Example:
[CODE]
// update creature_template set scriptname = “test” where entry = 123123;
class test : CreatureScript
{
public:
test() : CreatureScript(“test”)
struct someai : public CreatureAI
{
someai(Creature* creature) {}
void UpdateAI(uint32 diff) { }
};
CreatureAI* GetAI(Creature* creature)
{
if(creature->GetGUIDLow() == 123123)
return new someai(creature);
return NULL;
}
};[/CODE]
We are not allowed to assign guids in c++ scripts.
Well, whatever, but still you can use a C++ script, lua script, smart script etc with guids or entries or whatever.
You can for example code that the script does nothing or does different things depending on which guid the creature using the AI is.
This is what smart ai does when used on specific guids.
Can we spawn a guid of any entry and attach the script to it on spawn? Or does it have to be preassigned?
That’s quite wrong, it is not because that this is an educational project that we don’t have specific requirements.
We want to be blizzlike, so to create content and in the same way it makes us learn, it doesn’t mean that there are not any easier ways to do it.
Yes. This is possible also with the current C++ script AIs.
There is a function called SetAI or AIM_initialize or similar, that allows this as long as you have the creature object.
This means that you could change, remove and save the creature AI for a creature ingame at will with a command.
Though. Such should probably be only for development purposes.
Lets consider saved variables and so. Usually scripts assume things from the creature, the environment and what has happened and will happen (timed events etc)
Then you are a wow server “emulator” project, not an academic exercise giving a try at creating an MMORPG framework.
Which is my point exactly.
@Rochet2 is there a Lua stack for each Map/Instance or there is a global one ? If it’s shared across maps I wonder how thread-safe it is, especially when Maps are storing variables so writing to the same object at same time. Is there some kind of synchronization context involved ?
https://github.com/ElunaLuaEngine/ElunaTrinityWotlk/blob/master/dep/lualib/llimits.h#L155
#if !defined(lua_lock)
#define lua_lock(L) ((void) 0)
#define lua_unlock(L) ((void) 0)
#endif
oh… is that the locking mechanism to keep the Lua stack thread-safe ?
https://gist.github.com/jackpoz/2951de557d72adffa978 this is the race condition in Eluna
worldserver.exe!lua_rawgeti(lua_State * L, int idx, int n) Line 652 C
worldserver.exe!Eluna::RemoveRef(const void * obj) Line 314 C++
worldserver.exe!Spell::~Spell() Line 619 C++
worldserver.exe!Spell::scalar deleting destructor'(unsigned int) C++ worldserver.exe!SpellEvent::~SpellEvent() Line 6546 C++ worldserver.exe!SpellEvent::scalar deleting destructor’(unsigned int) C++
worldserver.exe!EventProcessor::Update(unsigned int p_time) Line 50 C++
worldserver.exe!Unit::Update(unsigned int p_time) Line 327 C++
worldserver.exe!Player::Update(unsigned int p_time) Line 1596 C++
worldserver.exe!Map::Update(const unsigned int t_diff) Line 667 C++
worldserver.exe!MapUpdateRequest::call() Line 44 C++
worldserver.exe!MapUpdater::WorkerThread() Line 116 C++
worldserver.exe!std::_Pmf_wrap<void (__cdecl MapUpdater::)(void) __ptr64,void,MapUpdater>::operator()(MapUpdater * _Pfnobj) Line 1231 C++
worldserver.exe!std::_Bind<1,void,std::_Pmf_wrap<void (__cdecl MapUpdater::)(void) __ptr64,void,MapUpdater>,MapUpdater * __ptr64>::_Do_call<,0>(std::tuple<> _Myfargs, std::_Arg_idx<0> __formal) Line 1150 C++
worldserver.exe!std::_Bind<1,void,std::_Pmf_wrap<void (__cdecl MapUpdater::)(void) __ptr64,void,MapUpdater>,MapUpdater * __ptr64>::operator()<>() Line 1138 C++
worldserver.exe!std::_LaunchPad<std::_Bind<1,void,std::_Pmf_wrap<void (__cdecl MapUpdater::)(void) __ptr64,void,MapUpdater>,MapUpdater * __ptr64> >::_Run(std::_LaunchPad<std::_Bind<1,void,std::_Pmf_wrap<void (__cdecl MapUpdater::*)(void),void,MapUpdater>,MapUpdater > > * _Ln) Line 196 C++
worldserver.exe!std::_LaunchPad<std::_Bind<1,void,std::_Pmf_wrap<void (__cdecl MapUpdater::)(void) __ptr64,void,MapUpdater>,MapUpdater * __ptr64> >::_Go() Line 188 C++
msvcp120d.dll!_Call_func(void * _Data) Line 28 C++
msvcr120d.dll!_callthreadstartex() Line 376 C
msvcr120d.dll!_threadstartex(void * ptd) Line 359 C
kernel32.dll!BaseThreadInitThunk() Unknown
ntdll.dll!RtlUserThreadStart() Unknown
Depends on the implementation. In our implementation threads are limited to 1 and there is 1 state. This way all of that is avoided.
However I am sure you are more interested on a multihreading model.
And as discussed in the other thread, there came up some models like state per map or thread etc.
It has been said that state per thread (at least) is the only sane way of handling multiple threads with lua.
With the current model of having the object pool it is per state.
This is problematic in a multithreaded situation if the invalidation model is kept the same, as multiple objects try to delete themselves from all lua states at random times.
The cure for that would be the second model I told earlier in this thread where we would invalidate all pointers on every top level hook call ending.
The way I have been imagining a threaded environment is a state per map, one global state.
States can send messages to eachother through locked queue with functions implemented. it would trigger the messages when the map updates are halted, when world update runs.
The data model would be as we use now, having an object pool of pushed objects in lua (this seems to be preferred, memory saving way among implementations and posts I have seen & read)
However, instead of invalidating objects when they are deleted, the earlier posted model of data invalidation at top level hook would be used to make pointers not crash. This will ofcourse force reinstanting the userdata on every hook call.
It would be better in my mind not to worry about invalid pointers and instead worry about invalid code as I noted in my earlier post.
In that model states can communicate safely, there is GC of lua userdatas, safety of pointers is as decided, the threads wont mess up eachother’s environments.
However. It does, for safety, limit all global functions and hooks, that for example get a player from another map, to the global state.
And it most likely requires certain functions to be coded to help maintain thread safety.
Lets say you want to send a message to everyone in the world, instantly.
There could be a function that calls a function for all players, in a locked loop.
local function printname(player)
print(player:GetName())
end
DoForPlayers(printname)
While this could be unsafe if abused, it could be alright in my mind so there is not much disabled functionality.
However I would also consider maybe for starters forcing global loops like that to be fired on the global state, which is always safe to use.
In that case there could be a function coded for the state messaging that allows functions to be called on the global state like this:
– global state code
function on_state_message(mapid, instanceid, msg)
dostring(msg)
end
RegisterServerEvent(ON_STATE_MESSAGE, on_state_message)
– some other state code
local msg = [[
for k,v in ipairs(GetPlayersInWorld()) do
print(v:GetName())
end
]]
SendStateMessage(-1, -1, msg)
With lua it is possible to pass around any value basically (number, string, table, function) from a lua state to lua state.
I coded that in an addon of mine, however as Eluna was using lua 5.2, I was not able to get the functions working and not sure if that is ultimately possible yet.
https://github.com/Rochet2/AIO
Yes, it is. Im unsure how they exactly intended it to be used. We dont use that though.
It has been said that implementing locks there is not enough. There are some, maybe obvious, problems in using a single lua state in a multithreaded environment even with locks implemented.
Thanks for that.
This is exactly why I agree with dfighter on the pointer management.
And why I would consider of not having a system at all to handle dead pointers.
Probably need to remove the current system in that light then : /
Might try to work on the other model.
ps. So long. Just so long …
I added the callstack of a race condition in the post above, here’s the link to the callstack anyway https://gist.github.com/jackpoz/2951de557d72adffa978
It shows how each Map thread can access the Lua stack in an unsafe way not thread-safe way.
This isn’t related to dead pointers at all tbh…
I didn’t get into discussing race conditions, since I don’t use Trinity, and I have no knowledge of it’s threading model therefore I couldn’t say a word about the topic.
It is.
The mechanism handling them is crashing.
The way it works is a set amount of threads handle all map update ticks in a locked queue of who gets there first.
Other threads include the global thread that handles looping all the other code every world tick and then the freeze detector etc.
This is my understanding.
The whole point of a scripting engine is that the scripter can do whatever he wants without causing any trouble (crash/memory leak/race condition). A crashing scripting engine is ok for personal use but not for an Open Source project or commercial product (just look at all crashfixes in GMod changelog happening in weird rare conditions)
What do you think about the other idea then, or do you have some of your own perhaps?
I was curious to see how this Eluna project tried to tackle the same issue I had to face 3 years ago. I consider race conditions http://en.wikipedia.org/wiki/Race_condition (which don’t cause crashes, they cause undefined behavior) not acceptable and at the same time mutexes quite overkill, especially if hooked up on every object destructor, making worldserver pretty much single threaded.
The multi state idea sounds interesting, keeping in mind as you said that some events MUST be executed in the safe global context and not in Map threads. I would still pay attention to handle cases when something moves from 1 map to another (Player, Aura, Spell) so that no race condition happen and no bad pointers remain around.