Hi,
I'm having additions to the wolf3d repository after more than 6.5 years, covering two added revisions of code. But there are also a few more general points to add:
- The various repositories aren't submodules of the "gamesrc-ver-recreation" repository itself anymore. Nuke.YKT and I haven't really been using this feature, so I've decided to do this change. The repositories still exist, I just don't use the submodule feature.
- While currently done in the wolf3d tree only, doing so in other repositories isn't necessarily out of the question. So, a significant subset of the file notes-restoration.md was replaced with a new README.md file, aiming to replace it. It should be much smaller, albeit it's still not necessarily a small file. I left various notes under notes-restoration.md but I currently have no guarantee of them being up-to-date. Interestingly, the older incarnation titled "game-srccode-ver-recreation" had a quite small README.md file added to it, so this can be seen in part as going back to the roots.
As for wolf3d, before getting to the aforementioned two code revisions, I also made these changes:
- Output build directories were collapsed. That should be more consistent with other repositories, the Blake Stone repository being just one example. For instance, WL1AP10.PRJ's output dir was changed from "OBJ\WL1AP10" to "WL1AP10".
- One directory was mistakenly named "STATIC\WOLF3D\WL920512". Instead of "WL920512", "WL920312" is used now.
Let's get to the added revisions themselves.
- The first one is what I suspect to be a quite unknown April 16 1992 prototype executable, somewhere in-between WL920312 and shareware v1.0. There wasn't a lot of new code added but I did have to go through existing macros, so it took some time. There were a few exceptions here and there, like placeholder code under the function "Victory", but what was there beforehand could be reused otherwise.
- Secondly, under a new directory, I added a separate modification of the open-source release of Wolf3D into code more-or-less matching a very early ROTT prototype. Known as ROTT0993, it's still essentially a modified Wolf3D. There wasn't a lot of code added and I didn't introduce new game data to the repository. This revision draws textured floors and ceilings and also has additional hotkey checks for debugging. As for the output EXE, expect differences in debugging symbols. I was reducing them, but even after making timestamps match, I still had 29 differing bytes.
To finish, here's one more thing. Getting back to the wolf3d repository after about 6.5 years, short of the few occasional edits done in-between, is not something I recall doing beforehand under gamesrc-ver-recreation right now. One point was known to me earlier, but having another look after a while further clarified it. Basically, the use of pre-processor macros did make it possible to cover many versions under a single codebase - currently 19 in total. On the other hand, over time, it may become difficult to maintain this code and track it with the added pre-processor macro checks. Then again, they did make it quite convenient to see differences between versions under a single window, without using a diff program. But that can be a challenge when it comes to supporting multiple versions in source ports. In ReflectionHLE's case, I mostly bypassed it by repeatedly rebuilding ported Wolf3D sources with a bit different configurations and then linking the builds into a single binary. Definitions of pre-processor macros inherited from gamesrc-ver-recreation were changed across these sets of objects. I still made adjustments, if due to the replacement of the macro UPLOAD with a variable of the same name, or any other technical reason.
Restoration of differing games' EXE revisions, Doom engine included
Re: Restoration of differing games' EXE revisions, Doom engine included
I'll start by quoting Nuke.YKT first:
1. Work on Raptor v1.2 was started by Nuke.YKT after receiving original files from Scott Host of Mountain King Studios. I joined later with the efforts, albeit I consider most work to still be done by Nuke.YKT. v1.1 was indeed covered by Nuke.YKT during the span of a single day, more-or-less. For v1.0, I helped a bit, mostly with Sound Blaster related code in DMX, but it was otherwise Nuke.YKT's work again. What we ended with were 3 separate private git branches representing versions 1.0, 1.1 and 1.2. They weren't ready for becoming public, due to the presence of proprietary DMX code and other possible aspects of code review. I eventually took the open-source release of v1.2 from skynettx' repository and added versions 1.0 and 1.1 to the same codebase, while preparing additional files (like Raptor's makefile and helper scripts) for support of all covered versions. I also added README.md, partially based in form on notes previously written for other games.
2. One more thing is the time span, given that it all started back in November 2021. While originally preparing the open-source release of Raptor v1.2, we were waiting to see if we'd maybe be able to use DMX. There were also internal pending code review notes. At some point, Scott Host seemed to want something a bit different; While he'd be working on a remaster of Raptor without a legacy mode, Nuke.YKT would work on a more direct port of the original game, with the idea that - possibly as an alternative to the DOS sources - the port itself would be the one to go open-source. Eventually, as written by Nuke.YKT, he had finally gotten back to Raptor v1.0 source reconstruction and finished it in September 2023. Host showed support for the original plan again, and not much later, skynettx published the sources.
3. I'll finish with the topic of forum threads. People have maybe noticed that the Duke4.net and/or Doomworld threads on the topic were new ones. In fact, Nuke.YKT already started a separate related Doomworld thread earlier, covering reconstructed Doom and Strife sources. Beforehand, there was generally just a single thread of mine about gamesrc-ver-recreation in each forum which had any; For a portion of these threads, each of them would be mostly specific to one game or engine, but that wasn't the case for all of them. I had actually considered deprecating my Doomworld thread covering Heretic+Hexen and migrating to Nuke.YKT's thread about Doom+Strife as a new general one. However, it was eventually decided with Nuke.YKT to start a new one for Raptor (at least on Duke4.net and Doomworld for now). A few reasons to bring up:
- If a post about a newly covered game (e.g., Raptor) were to be made in a more general existing thread, then people who see the title wouldn't immediately be aware of Raptor being the topic; They wouldn't know without inspecting the thread.
- As an added post to a previously created thread, chances are it would look like a small update, rather than something entirely new.
- Finally, I think that usually, one person's reverse-engineering projects covering different code-bases (e.g., games) are considered to be separate endeavors. Sure, we've had varying repositories under gamesrc-ver-recreation, and there's more than enough to share (like the Apogee Sound System). But I still think the notion of such projects being separate makes sense.
Now, adding my words as well.Nuke.YKT wrote:Hi,
To celebrate 30th anniversary of Raptor, gamesrc-ver-recreation project now covers all known EXE revisions of Raptor. As a reminder, gamesrc-ver-recreation is a project which aims to reconstruct EXEs of various games accurately as possible using their released source codes as a base. Like with Doom engine games EXE restorations, this release does not include proprietary DMX code.
This project started in November 2021, when copy of source code of Raptor and parts of surrounding code (GFX library) were received from original developer, Scott Host. Using code we got and copy of DMX code we already had, me and NY00123 starting piecing them together to get functioning DOS exe. Eventually we managed to compile it, but as expected, there were few differences compared to original v1.2 EXE. With some more effort we found and fixed all the differences, and got EXE that matched original EXE byte-by-byte. Using this, I immediately started reconstructing sources for earlier releases. v1.1 release was reconstructed fairly easily within a day or so. Reconstruction of v1.0 though, proved to be quite difficult, as the DMX version used in this version was much earlier than any version we had (for example, Gravis ultrasound code was completely different, and still was using hardware mixing like Doom 1.3 and earlier did). We weren't able to finish this at the time. Shortly after this we moved to the Doom and Strife source code restoration, and DMX restoration we did for Raptor v1.2, turned out to be useful there. Specifically, we found out that Doom 1.7/1.7a uses exact same DMX revision.
Much later though, in September 2023, I was trying to reconstruct Doom 1.2 executable. As it uses DMX version quite close to Raptor 1.0, I decided to look at this again. This time I had much more luck, and thus Raptor v1.0 finally was reconstructed.
Shortly after this, branch of v1.2 reconstruction was published on github by skynettx as official GPL release of Raptor source code. Based on this, NY00123 put together this repository for public release.
https://bitbucket.org/gamesrc-ver-recreation/raptor
1. Work on Raptor v1.2 was started by Nuke.YKT after receiving original files from Scott Host of Mountain King Studios. I joined later with the efforts, albeit I consider most work to still be done by Nuke.YKT. v1.1 was indeed covered by Nuke.YKT during the span of a single day, more-or-less. For v1.0, I helped a bit, mostly with Sound Blaster related code in DMX, but it was otherwise Nuke.YKT's work again. What we ended with were 3 separate private git branches representing versions 1.0, 1.1 and 1.2. They weren't ready for becoming public, due to the presence of proprietary DMX code and other possible aspects of code review. I eventually took the open-source release of v1.2 from skynettx' repository and added versions 1.0 and 1.1 to the same codebase, while preparing additional files (like Raptor's makefile and helper scripts) for support of all covered versions. I also added README.md, partially based in form on notes previously written for other games.
2. One more thing is the time span, given that it all started back in November 2021. While originally preparing the open-source release of Raptor v1.2, we were waiting to see if we'd maybe be able to use DMX. There were also internal pending code review notes. At some point, Scott Host seemed to want something a bit different; While he'd be working on a remaster of Raptor without a legacy mode, Nuke.YKT would work on a more direct port of the original game, with the idea that - possibly as an alternative to the DOS sources - the port itself would be the one to go open-source. Eventually, as written by Nuke.YKT, he had finally gotten back to Raptor v1.0 source reconstruction and finished it in September 2023. Host showed support for the original plan again, and not much later, skynettx published the sources.
3. I'll finish with the topic of forum threads. People have maybe noticed that the Duke4.net and/or Doomworld threads on the topic were new ones. In fact, Nuke.YKT already started a separate related Doomworld thread earlier, covering reconstructed Doom and Strife sources. Beforehand, there was generally just a single thread of mine about gamesrc-ver-recreation in each forum which had any; For a portion of these threads, each of them would be mostly specific to one game or engine, but that wasn't the case for all of them. I had actually considered deprecating my Doomworld thread covering Heretic+Hexen and migrating to Nuke.YKT's thread about Doom+Strife as a new general one. However, it was eventually decided with Nuke.YKT to start a new one for Raptor (at least on Duke4.net and Doomworld for now). A few reasons to bring up:
- If a post about a newly covered game (e.g., Raptor) were to be made in a more general existing thread, then people who see the title wouldn't immediately be aware of Raptor being the topic; They wouldn't know without inspecting the thread.
- As an added post to a previously created thread, chances are it would look like a small update, rather than something entirely new.
- Finally, I think that usually, one person's reverse-engineering projects covering different code-bases (e.g., games) are considered to be separate endeavors. Sure, we've had varying repositories under gamesrc-ver-recreation, and there's more than enough to share (like the Apogee Sound System). But I still think the notion of such projects being separate makes sense.
Re: Restoration of differing games' EXE revisions, Doom engine included
Looks like a good time for a new post.
1. First of all, after about 11 years and 5 months of hosting on Bitbucket, a decision was made to move gamesrc-ver-recreation's sources to GitHub. That follows Bitbucket-side policy changes. One is about forking a repository outside of its workspace, even if less relevant for gamesrc-ver-recreation so far. That policy change might be related to having companies' use of Bitbucket workspaces in mind. Another policy change is the cleanup of free workspaces which are unused after some time period.
Here's the new URL: https://github.com/gamesrc-ver-recreation
In fact, this endeavor briefly started as a GitHub-hosted repository named "game-srccode-ver-recreation", as shown by a git remote URL of an old local repository. It had its life in 8 commits during November 2014's first week.
2. Secondly, both Nuke.YKT and I had a few updates for the Build Engine repository, albeit actual code changes committed by me are mostly based on code received from Ken privately and released with permission in 2024*.
About a year ago, Nuke.YKT made changes allowing the recreation of Build Engine object files matching multiple versions of Blood and its Build editor, MapEdit. Covered matching versions, not including the 3dfx patch, are 0.91-1.21.
As for me, I used engine-side C code released in 2024 in order to make some changes, say cleanups. At times, it was just about making code look closer to the original. That said, C code from buil062896.zip helped me remove a couple of hacks from MMULTI.C, while code from BUILDBAK.ZIP helped fixing incorrect local variable naming in CACHE1D.C's functions "compress" and "uncompress".
Most important, BUILDBAK.ZIP and buil1995.zip helped me fix inaccuracies in binary code recreation within 4 functions of ENGINE.C. In particular, an ENGINE.OBJ file fully matching the Build Editor (BUILD.EXE) for Duke3D v1.3d in binary code may be recreated now (along with the rest of the Build Engine objects). Ignoring a few caveats like possible environment-dependent data inserted between C string literals by the Watcom 10.0 series of compilers, all code of this revision of BUILD.EXE may be accurately recreated now.
3. Last but not least, Nuke.YKT also got something up within the Apogee Sound System (audiolib) repository. In addition to the Build Engine, during the same month of April 2025, he applied changes allowing the recreation of code matching version 1.12 as used in Blood. People might remember that audiolib's open-source release is also identified as 1.12, but it turned out to have at least a few differences. As a consequence, I decided to add the option to build code matching two AUDIO_WF.LIB files from the open-source release of 2002, as well as that release's C code.
To finish, last March, Nuke.YKT improved upon my earlier work from 2017 Q4. I was working on recreating v1.04 as used in multiple DOS versions of ROTT identified as v1.3, but had major difficulties with the function MV_ServiceVoc. Thanks to the assistance of line number information found in the original AUDIO_WF.LIB file's MULTIVOC.OBJ, Nuke.YKT managed to change the function body, so it may be accurately recreated now. On a minor note, back in February 2019, I added the option to more-or-less recreate v1.09 as used in Shadow Warrior (1997) 1.0-1.2. That said, the contents of the generated FX_SetupCard differed by one register read. That is fixed by Nuke.YKT as well now - probably thanks to two switch statement's cases having separate identical code bodies, instead of one shared body of code.
* See video and URLs referenced in video description: https://www.youtube.com/watch?v=xdJfv5e677Y
1. First of all, after about 11 years and 5 months of hosting on Bitbucket, a decision was made to move gamesrc-ver-recreation's sources to GitHub. That follows Bitbucket-side policy changes. One is about forking a repository outside of its workspace, even if less relevant for gamesrc-ver-recreation so far. That policy change might be related to having companies' use of Bitbucket workspaces in mind. Another policy change is the cleanup of free workspaces which are unused after some time period.
Here's the new URL: https://github.com/gamesrc-ver-recreation
In fact, this endeavor briefly started as a GitHub-hosted repository named "game-srccode-ver-recreation", as shown by a git remote URL of an old local repository. It had its life in 8 commits during November 2014's first week.
2. Secondly, both Nuke.YKT and I had a few updates for the Build Engine repository, albeit actual code changes committed by me are mostly based on code received from Ken privately and released with permission in 2024*.
About a year ago, Nuke.YKT made changes allowing the recreation of Build Engine object files matching multiple versions of Blood and its Build editor, MapEdit. Covered matching versions, not including the 3dfx patch, are 0.91-1.21.
As for me, I used engine-side C code released in 2024 in order to make some changes, say cleanups. At times, it was just about making code look closer to the original. That said, C code from buil062896.zip helped me remove a couple of hacks from MMULTI.C, while code from BUILDBAK.ZIP helped fixing incorrect local variable naming in CACHE1D.C's functions "compress" and "uncompress".
Most important, BUILDBAK.ZIP and buil1995.zip helped me fix inaccuracies in binary code recreation within 4 functions of ENGINE.C. In particular, an ENGINE.OBJ file fully matching the Build Editor (BUILD.EXE) for Duke3D v1.3d in binary code may be recreated now (along with the rest of the Build Engine objects). Ignoring a few caveats like possible environment-dependent data inserted between C string literals by the Watcom 10.0 series of compilers, all code of this revision of BUILD.EXE may be accurately recreated now.
3. Last but not least, Nuke.YKT also got something up within the Apogee Sound System (audiolib) repository. In addition to the Build Engine, during the same month of April 2025, he applied changes allowing the recreation of code matching version 1.12 as used in Blood. People might remember that audiolib's open-source release is also identified as 1.12, but it turned out to have at least a few differences. As a consequence, I decided to add the option to build code matching two AUDIO_WF.LIB files from the open-source release of 2002, as well as that release's C code.
To finish, last March, Nuke.YKT improved upon my earlier work from 2017 Q4. I was working on recreating v1.04 as used in multiple DOS versions of ROTT identified as v1.3, but had major difficulties with the function MV_ServiceVoc. Thanks to the assistance of line number information found in the original AUDIO_WF.LIB file's MULTIVOC.OBJ, Nuke.YKT managed to change the function body, so it may be accurately recreated now. On a minor note, back in February 2019, I added the option to more-or-less recreate v1.09 as used in Shadow Warrior (1997) 1.0-1.2. That said, the contents of the generated FX_SetupCard differed by one register read. That is fixed by Nuke.YKT as well now - probably thanks to two switch statement's cases having separate identical code bodies, instead of one shared body of code.
* See video and URLs referenced in video description: https://www.youtube.com/watch?v=xdJfv5e677Y