Reverse engineering (was:Temporary modding) thread

Talk about creating Grimrock 1 levels and mods here. Warning: forum contains spoilers!
Post Reply
ymgve
Posts: 6
Joined: Thu Apr 12, 2012 7:59 pm

Reverse engineering (was:Temporary modding) thread

Post by ymgve »

Since the modding forum isn't open yet, I thought I might place this here. I've worked a bit on dissecting the Grimrock.dat file, and I've got it mostly figured out - only downside is that the devs uses hashes of filenames instead of the filenames themselves, so it's hard to find out what the files are actually named.

Here's an extractor, written in Python: http://pastebin.com/w6anxSjY
Incomplete list of file names: http://pastebin.com/hupic6F4

But you might not be interested in that, you are probably more interested in custom levels. So far I've only found a very bad and stupid method of running custom levels:
1. Make a backup copy of your Grimrock.dat file
2. Use a hex editor to search for the hex bytes AF F4 D1 88 in Grimrock.dat and replace them with 00 00 00 00 (This makes the game unable to find the first level inside the .dat and forces it to search the file system instead.
3. Make the directories "assets\dungeons\grimrock" in your Grimrock game dir, and place the level named as level01.lua there.
4. Start up the game, start a new game, and play.

To get started, here's the manually decompiled level 1 map, which shows how some things are made: link removed
Last edited by ymgve on Mon Apr 16, 2012 7:34 pm, edited 1 time in total.
askjosh
Posts: 11
Joined: Wed Apr 11, 2012 6:44 pm

Re: Temporary modding thread

Post by askjosh »

I cant wait to start modding however I will have to wait for the proper tools to come out because most of what you just said sounds like greek to me. With that being said I am glad there are others like you that diving right in :-)
semi
Posts: 4
Joined: Fri Apr 13, 2012 4:43 pm

Re: Temporary modding thread

Post by semi »

Awesome work on the extractor so far! I've experimented with it a little bit more to get some additional information:

I've extracted all the Lua bytecode files from grimrock.dat (identified by their header 1B 4C 4A 01) and ran GNU Strings over those to find possible paths. From manually looking over the generated strings, I've seen that most of them are asset/foo/bar.baz paths, so i filtered those out by regexes.

Here is the list of asset paths (851 entries), plus one that I filtered by which assets could actually be found (277 entries) in the .dat file.

For the lazy, here is a list that will extract all levels.


Edit:
I've extracted all files just by their hashes and ran my path search on those files again. This leads to a list of 1484 paths, 381 of which can be resolved into actual files.

Edit 2:
The good people at the XeNTaX forum have made quite a bit of progress. Amongst other things, they suggest that LoG uses a context processor that will transform certain suffixes so that multiple files can be loaded by one file name. e.g.:
  • mymodel.fbx -> mymodel.animation, mymodel.mesh, mymodel.model
  • myimage.tga -> mymodel.d3d9_texture
I've combined that information with the input.txt included with their .dat file extractor and the asset paths found by string search. That gives us a lot more resolved filenames - we can now resolve 2069 files, that's all but 264! (download file lists here)
Last edited by semi on Fri Apr 13, 2012 9:06 pm, edited 1 time in total.
ymgve
Posts: 6
Joined: Thu Apr 12, 2012 7:59 pm

Re: Temporary modding thread

Post by ymgve »

It appears that if the game can't find a .png file, it will try again with the extension replaced by .d3d9_texture - that should give some more valid files.
sage2
Posts: 16
Joined: Thu Apr 12, 2012 1:32 am

Re: Temporary modding thread

Post by sage2 »

Will the game preferentially load loose .lua files?
semi
Posts: 4
Joined: Fri Apr 13, 2012 4:43 pm

Re: Temporary modding thread

Post by semi »

See above, I already stumbled upon that information. I've posted a new file list for you. Got to cook some dinner now :)
User avatar
Crashbanito
Posts: 326
Joined: Sun Apr 01, 2012 2:50 pm
Location: Louisiana, USA
Contact:

Re: Temporary modding thread

Post by Crashbanito »

I replaced all the snails with spiders in level 1. Went and got something to eat. Started a new game and nearly gave myself a heart attack. :lol:
Grimrock FAQ | If you see something fishy, flag that post! | My Gaming/Tech Blog
ymgve
Posts: 6
Joined: Thu Apr 12, 2012 7:59 pm

Re: Temporary modding thread

Post by ymgve »

Looking through the .lua files I see something that hints at a developer console being in there. It appears to be accessed by pressing "§" (shift+tilde on US keyboards) and having a "developer = true" line in your grimrock.cfg. Though I still can't get it to work. Possibly disabled?
semi
Posts: 4
Joined: Fri Apr 13, 2012 4:43 pm

Re: Temporary modding thread

Post by semi »

ymgve, how are you looking through the lua files? Did you find a working disassembler that you are hiding from us? I'd like to have a look, too :)
ymgve
Posts: 6
Joined: Thu Apr 12, 2012 7:59 pm

Re: Temporary modding thread

Post by ymgve »

Yeah - http://luajit.org/ - compile this, then you can use luajit.exe -bl filename.lua to get a bytecode listing. If you can't compile it yourself, I've tried making some precompiled binaries:

http://dl.dropbox.com/u/1593421/luajit.rar

I'm now trying to run the game with just loose files (by creating a dummy grimrock.dat only consisting of hex 4752415200000000), but I'm now stumped - I constantly get a "File not found: assets/shaders/crystal.hlsl" error and can't figure out what the "translated" file name should be.
Post Reply