loadlib (lua's DLL import function) - crash on function call *SOLVED*

[violgamba]

I know that this is a long shot, but I was wondering if anyone had an idea about a crash I'm seeing when playing with the nutcracker release.

I've been experimenting with lua's (built-in) "loadlib()" function. loadlib() lets you import a DLL file that contains functions written in C with a particular protocol called "lua_CFunction". The C functions, once imported into lua, can be treated like regular lua functions.

Btw, I know that this would break the "Modding and Asset usage terms" if distributed.

So, I've been able to import a DLL and run it's functions from the lua 5.1 console application. And I can import the DLL into Grimlock 2, and also print the imported function's tostring value (or nil, if the import failed). However, when I try to run any of the imported functions, Grimlock 2 crashes. No error message, just the crash.

I just wonder if there's a known explanation or, better yet, a solution for this crash. I think I've read that Grimrock's lua engine isn't vanilla, which would certainly explain it.

EDIT: I just successfully called an "idle" function, which takes no parameters and returns nothing. The crashing is definitely dependent on taking and/or returning a value. Most likely the "lua_to..." and "lua_push..." function families. FYI, I made sure that the dll is 32 bit, strings are multibyte (not unicode) and lua 5.1 is used.

Also, handling numbers crashes Grimrock 2, but handling strings freezes Grimrock 2 instead (with the music still playing).

[minmay]

I think I've read that Grimrock's lua engine isn't vanilla, which would certainly explain it.

It's LuaJIT 2.0.0-beta9 with no modifications. I would strongly recommend using LuaJIT's ffi library instead of loadlib, I've used it a bit in Grimrock 2 and found no issues with it. Arguments and return values certainly work!

[violgamba]

Thank you for that useful information! It definitely advanced the situation, and It almost seemed to resolve the issue.

First I tried compiling the dll with "Luajit 2.0.0 beta 9" instead of "Lua 5.1". That seemed to resolve my issue with "loadlib()", but another cropped up as I'll explain in a minute. So I switched gears to the ffi library.

==============================================

Ffi worked fine in the Luajit 2.0.0 beta 9 CONSOLE. Here is the working code ("tstDll.dll" is a simplification of the absolute path used).

Code: Select all

	ffi = require("ffi")
	extern_tst = ffi.load("tstDll.dll", "getInt")
	ffi.cdef([[int getInt();]])
	print("??", extern_tst.getInt(), "!!")

I then added this code to a method in the Grimrock 2 game (replacing "print" with "console:print"). FYI, I'm using "NewGameMenu:init", so the code is run whenever I click on the "New game" menu on the title screen. Anyway, the code crashed Grimrock 2 with this error:

attempt to call global 'require' (a nil value)

When I commented out the "require" line, the error became:

attempt to call global 'ffi' (a nil value)

These errors suggest that "require" is unavailable in the main game and, thus, ffi is unavailable in the main game.

In the umod pack code, the "require()" function is used, but only in "DungeonEditor.lua".

==============================================

As for "loadlib()"...

Luajit 2.0.0 beta 9 CONSOLE crashed on running the imported DLL functions, just like Grimrock 2 did. This suggested that I was seeing the same issue in Grimrock 2 and the Luajit console. I then adjusted the DLL to build with "Luajit 2.0.0 beta 9" instead of "Lua 5.1". After this, the console ran the imported DLL functions just fine. Promising!

However, now Grimrock 2 fails to import any functions from the DLL (they register as as nil). Argh!

Experimentation shows that Grimrock 2 will import any DLL functions (even those without the "lua_CFunction" protocol), unless any of the DLL functions call any of luajit's functions ("lua_pushnumber", "lua_gettop", etc). If any of the DLL functions call these luajit functions, then none of the DLL functions will import properly (all register as nil).

This is especially mystifying as the same DLL functions import fine in the Luajit 2.0.0 beta 9 CONSOLE.

==============================================

Again, thanks for your earlier suggestions. Any thoughts on these issues would be appreciated.

[minmay]

attempt to call global 'require' (a nil value)

When I commented out the "require" line, the error became:

attempt to call global 'ffi' (a nil value)

These errors suggest that "require" is unavailable in the main game and, thus, ffi is unavailable in the main game.

Whatever function you're trying to do this in is in the wrong environment. You'll have the same issue with loadlib. Do it from a function whose environment is _G.
You can change the environment of an existing function with setfenv(), though obviously this also has to be done from a function that already has access to _G.

[violgamba]

SOLVED - at least for loadlib

ISSUE:
When building "Luajit 2.0.0 beta 9", the build script built lua51.lib, which I linked to when creating my DLL file, but it also created lua51.dll, which I didn't notice. This meant that lua51.lib was really just a stub for lua51.dll and, thus, my DLL was reliant on lua51.dll to work (though I didn't know it). The Luajit console was able to loadlib() my DLL because the lua51.dll happened to be in the same directory. Grimrock 2, was not able to loadlib() my DLL because lua51.dll was not in the same directory.

SOLUTION:
I re-ran the built script for "Luajit 2.0.0 beta 9", with the "static" parameter. This created a full lib file, rather than a DLL stub, so no reliance on lua51.dll. Now my DLL functions are successfully loadlib'd and running from Grimrock 2!

SUMMARY:
A DLL to be imported into Grimrock 2, using loadlib, must be built with "Luajit 2.0.0 beta 9" (not vanilla lua 5.1). ALSO, it's important that the Luajit library is STATICALLY linked into the DLL, not dynamically linked.

==============================================

Now, about ffi...

It'd be nice to get it working, since it's easier to write C functions for than loadlib().

I understand that the function I'm using ("NewGameMenu:init") is in the wrong environment. The proposed solution is to use setfenv(). I've not used that function before, but I see it being used in a few places in the umod pack. I'll experiment and report back.

[violgamba]

The ffi issue is now resolved as well, thanks to a workaround found by Minqmay.

PROBLEM
Turns out that access to the "require()"function has been removed for umods in the latest Grimrock 2 release.

SOLUTION
You can safely replace

Code: Select all

ffi = require("ffi")

with

Code: Select all

ffi = package.preload.ffi()

After that, ffi works as expected.