History of Oni modding: Difference between revisions

From OniGalore
Jump to navigation Jump to search
m (→‎Engine patching: adding link to the side story of the Mac betas, fixing other link)
m (link fix)
 
(40 intermediate revisions by the same user not shown)
Line 1: Line 1:
:''This article is a historical perspective on Oni modding; for an introduction to how to actually mod Oni, see [[Modding Oni|here]].''
{{Hatnote|This article is a historical perspective on Oni [[wp:Video game modding|modding]]; for an introduction to how to actually mod Oni, see [[Introduction to modding|here]]. To download mods, you can start [[AE|here]].}}
As has been documented on the page about [[Oni]]'s overall development, time ran out for Bungie West to do everything they wanted with this very ambitious game. Just as some features had to be omitted, the promise of modding support<ref>


As has been documented on the page about [[Oni]]'s overall development, time ran out for Bungie West to do everything they wanted with this very ambitious game. Just as some features had to be omitted, the promise of modding support<sup>1</sup> also failed to come true. However, even before Oni fans really knew that there would never be official modding tools from Bungie, they were already eagerly diving into the game data to see how they could mod it.<sup>2</sup> Nevertheless, the lack of any documentation of tools for modding undeniably slowed the development of mods within the community. Only for the last {{age|2007|11|14}} {{#ifexpr:{{CURRENTYEAR}}-(2007)-({{CURRENTMONTH}}<(11)or{{CURRENTMONTH}}=(11)and{{CURRENTDAY}}<(14))=1|year|years}} <!--This #ifexpr statement could be made into a template at some point, I just don't feel like it right now; the date that's hard-coded into that would-be template (11/14/07) is the date OniSplit started, and it seemed like as good a day as any to mark the point where our modding really started improving --Iritscen--> has the community's modding capabilities covered nearly all aspects of the game.
In [[Harry]]'s [http://oni.bungie.org/special/part1/brent.html interview], Brent Pease, the Project Lead, said: "What we will do is, I'm about 90 percent sure of this, is give away the file format." He also promises with equal certainty that the tools for converting 3D Studio Max models into Oni models will be given away. Pease eventually left the project before its completion. Hardy LeBel, his replacement, would later say, "[Due to the time crunch] our primary focus had to be on completing the content and features of the game rather than building in support for the mod community." Additionally, Bungie no longer owned the game by the time it shipped, so it would have been legally questionable to release header files from the source code, which would be the simplest and quickest way to "give away the file format".
</ref>
also failed to come true. However, even before Oni fans really knew that there would never be official modding tools from Bungie, they were already eagerly diving into the game data to see how they could mod it.<ref>


There are three basic areas in which the game can be modded. In order of difficulty they are: scripting, binary resources, and the engine code.  Along the way, we'll look at some key discoveries the community made in more detail.
Fans wasted no time in dumping the readable text from the executable and trying to figure out how to enable Developer Mode, starting before Oni's official release date by analyzing the demo<!--Dead link: http://carnage.bungie.org/oniresforum/onires.archive.pl?read=35 -->. Little did they know it would take another five years before this particular goal would be reached.
</ref>
Nevertheless, the lack of any documentation or tools for modding undeniably slowed the development of mods within the community. Only since 2010<!--Nov. 6, 2010 is the date that geyser posted about OniSplit supporting pathfinding grids for new levels in the article "New levels".--> has the community's modding capabilities covered nearly all aspects of the game.
 
There are three basic areas in which the game can be modded: scripting, binary resources, and the engine itself.  Along the way, we'll look at some key discoveries the community made in more detail.


==BSL scripting==
==BSL scripting==
[[Image:One Tiny Mistake.jpg|thumb|225px|right|In this script, Konoko pulls her gun on the secretary in the beginning of Chapter 2, an action she does not perform in the original cutscene. The script alters the level so as to tighten up the story as well as provide a new challenge for veteran players.]]
[[Image:One Tiny Mistake.jpg|thumb|225px|right|Konoko pulls her gun on the secretary in the beginning of Chapter 2, a change to the original cutscene made by a script which tightens up the story and provides a new challenge for veteran players.]]
Since the .bsl files were plain-text, and in plain <u>sight</u> within the folders that held Oni's game data, it's not surprising that fans quickly learned some basic commands by seeing what Bungie had written, and making their own scripts. Further commands were discovered by dumping readable text from the engine, although not everything worked as it seemed it should.
Since the .bsl files were plain-text, and in plain <u>sight</u> within the folders that held Oni's game data, it's not surprising that fans quickly learned some basic commands by seeing what Bungie had written, and making their own scripts. Further commands were discovered by dumping readable text from the engine, although not everything worked as it seemed it should.


As scripters were learning BSL, they produced scripts that at first merely modified the original level logic in minor ways (e.g. Titan Mode). Later, they produced more original, often epic fight scripts (e.g. "Multiple Mukades" and "[[OTA]]"); finally, some ambitious modders scripted totally new objectives and mission layouts (e.g., Mariachi Bear's alternate storyline). However, fully original scripts are all the more prone to bugs. The first two are easier to set up, but extreme situations tend to overload the engine (as in [[Blam]]!) in ways that the scripters have trouble preventing.
As scripters were learning [[BSL]], they produced scripts that at first merely modified the original level logic in minor ways. Then they produced more original, often epic fight scripts; later, some ambitious modders scripted totally new objectives and mission layouts (e.g., Mariachi Bear's alternate storyline). Some of the most complex scripts are the [[OTA]] scripts which were eventually developed to make up for a lack of [[Multiplayer|multiplayer gameplay]]. However, fully original scripts are all the more prone to bugs. The first two are easier to set up, but extreme situations tend to overload the engine (as in [[Blam]]!) in ways that the scripters have trouble preventing.


Testing the scripts required large numbers of attempts in launching Oni and running the scripts to see how they worked.  This process became much faster when the Developer Mode console was unlocked (see below) and individual commands could be tested easily.  While this is the easiest way to mod Oni, it's also the most limited; partly because of the lack of full documentation for BSL (which this wiki hopes to provide at some point), and partly because scripts are limited to the resources that an original level has to offer.
Testing the scripts required large numbers of attempts in launching Oni and running the scripts to see how they worked.  This process became much faster when the Developer Mode console was unlocked (see below) and individual commands could be tested easily.  While this is the easiest way to mod Oni, it's also the most limited; partly because of the lack of full documentation for BSL (which this wiki hopes to provide at some point), and partly because scripts are limited to the resources that an original level has to offer.
<br clear="all" />
{{clearall}}
 
==Binary resources==
==Binary resources==
===Private knowledge, public results===
[[Image:OniTools.jpg|thumb|225px|right|Ian Patterson was probably the most influential early binary hacker, and his trail-blazing program [[OniTools]] for the Mac was quickly ported to Windows.  It had support for delta patches, and the earliest binary mods were probably based on this method.]]
[[Image:OUP_txmpreplace.jpg|thumb|225px|right|OniUnPacker by Alloc, premiering in fall of 2005, provided a simple interface for finding the resource you wanted and displaying it, as well as replacing it.]]
[[Image:AE_Installer_1.jpg|thumb|225px|right|The mod installer released in 2009. For the first time, players could pick and choose what mods to install.]]
[[Image:AEI_main_window.jpg|thumb|225px|right|The Installer released in 2013 could update itself, the player's mods, and the patches made to the game. It was written from scratch in Java for ease of cross-platform support.]]
Fans with some expertise in parsing files with hex editors found that the level data files held four-letter codes, indicating the start of each resource within that file. Within a week of the street date for Oni, the community had its first resource modding tool, OniTools. While the author did not yet know how all the data worked, his program could at least parse it, and allow the modder to more easily edit each resource without having to count in hexadecimal. It also supported user-friendly replacement of texture maps and allowed in-line viewing of the game's 3D models. There seems to be little evidence that this knowledge was put to use in modding the game resources beyond simple mods such as texture replacements, no doubt due to the lack of a complete understanding of how Oni's dozens of resource types were inter-related.
This did not stop one intrepid programmer named Pierre Terdiman from deciding to import data from Oni into the game engine that he was writing from scratch. Called [[Konoko Payne]], it was a long-term project which took the form of (1) an underlying engine, and (2) a short scenario for it that resembles a sequel to Oni. In order to extract models and animations from Oni and recreate them precisely in his project, Pierre became one of the early pioneers in "reverse engineering" the game's resources, although his acquired knowledge remained largely private, as his work predated the community's organized public attempts at gathering this information.
===Hackers unite===
Perhaps the first place that knowledge of Oni's data was published online was on [http://oni.moltenstudios.com/tool/index.php Oni Master's site], which started as early as 2003 and was based on the knowledge that had gone into OniTools. Then, in 2005, [[User_talk:Ssg|Ssg]] opened a site with a much more thorough examination of the game data (the site is now hosted at http://ssg.oni2.net). Later that year, this pool of knowledge was painstakingly moved by [[User:Geyser|geyser]] and Ssg to its current home on the newly-opened wiki, OniGalore. The purpose of creating the [[OBD|Oni Binary Database]] was to allow collaboration, and soon [[User_talk:Neo|Neo]] appeared on the scene, adding his own knowledge that he had been acquiring independently until then, and collaborating on further investigations into the resource formats. Today at least 95% of the binary resource formats are documented byte for byte thanks to their hard work.
:''Further reading:'' [[History of the Oni community#2005|History of the Oni community (2005)]].
===Modding by candlelight===
Originally, modders had to alter these binary resources manually with hex editors to adapt them to their scripts. [[User:Loser|Loser]] was the first modder to manually edit the game data to go along with his scripting, with a massively reshuffled Warehouse. Loser's [[AE:Warehouse|Warehouse mod]] included a global component (modified level0_Final), level-specific binary modifications, and a set of customized scripts that took full advantage of the modified binaries.
Minor binary modding had already occurred before Loser's Warehouse (thus, while compiling the [[OBD]] database, Ssg had released a version of level0_Final in which all of Konoko's combos were unlocked). Still, it was the Warehouse mod released in 2005 which first showed the "full" potential of binary modding. It inspired a short-lived initiative called the [[AE:BIP|Binary Improvement Project]], aimed at providing upgraded binaries for all levels, not just level0_Final and level1_Final.


[[Image:OniTools.jpg|thumb|225px|right|Ian Patterson was probably the most influential early binary hacker, and his trail-blazing program [[OniTools]] for the Mac was quickly ported to Windows.  It had support for "CMPO" delta patches, and the earliest binary mods were probably based on this method.]]
BIP was hindered by large upload/download sizes as well as by the potential legal issues involved in distributing entire files from the game; in 2006 it was replaced by a smarter alternative to supplying whole levels, in the form of [[OUP|OniUnPacker]]-based patches and was termed [[AE:ONK|Oni ni Kanabô]]. The modder would edit his .dat/.raw files in OUP, and then release a single binary resource patch. This was both bandwidth-efficient and more legal-friendly, but suffered from one major limitation: the indexation and offsets of data vary a lot across the available versions of Oni. Thus, resources exported with OUP were not truly standalone or portable. Also, OUP still lacked the ability to import new resources alongside existing ones.
Fans with some expertise in parsing files with hex editors found that the binary data (i.e., the files ending in .dat and .raw, not to be confused with "binary" as a term for an application) held four-letter codes, indicating the start of each resource within that file.  Within a week of the street date for Oni, the community had its first resource modding tool. While the author did not yet know how all the data worked, his program could at least parse it, and allow the modder to more easily edit each resource without having to think in hexadecimal. It also supported user-friendly replacement of texture maps and allowed in-line viewing of the game's 3D models.


There seems to be little evidence that this knowledge was put to use in modding the game resources beyond simple mods such as texture replacements, no doubt due to the lack of a complete understanding of how Oni's 65 resource types were inter-related. As time went on, though, modders increasingly yearned for more control over the game than BSL could afford them.  Serious efforts in "hacking" the binary data began, as far as the modern community is concerned, in the fall of 2004 when [[User_talk:Ssg|ssg]] started a site on the newly-opened [http://www.oni2.net oni2.net]. A year later, this pool of knowledge was painstakingly moved by [[User:Geyser|geyser]] and ssg to its [[OBD|current home]] on the wiki to allow collaboration; at this point, [[User_talk:Neo|Neo]] appeared on the scene, adding his own knowledge that he had been acquiring independently until then.  Today at least 90% of the binary resources are documented byte for byte thanks to their hard work.
===Modular modding===
The advent of Neo's [[OniSplit]] in late 2007 marked a radical improvement to our modding methods. A command-line tool that runs on Windows and Macs, OniSplit breaks Oni's resources into standalone files, making mods truly portable across different retail versions of Oni. These resources can be exported to standard formats as text, images, sounds and 3D models, edited, and recombined into the level data files that Oni expects, and new resources can also be created from scratch. To take advantage of this revolution in modding, geyser began preparing an OniSplit-based port of some of the work done on BIP and ONK, which he hoped would be ready for the seventh anniversary of Oni's release in January 2008.


Originally, modders had to alter these binary resources manually with hex editors to adapt them to their scripts. [[User:Loser|Loser]] was the first modder to manually edit the game data to go along with his scripting, with a massively reshuffled Warehouse. Loser's Warehouse mod included a global component (modified level0_Final), level-specific binary modifications, and a set of customized scripts that took full advantage of the modified binaries.
This "[[AE|Seventh Anniversary Edition]]" had major releases in January and July 2008. The tradition continued of using a central modding project to organize a general revision of Oni's content, as seen on pages like [[AE:Ninja]]. The project then changed hands to [[User:Gumby|Gumby]], who designed a framework and a GUI-based installer that allowed players to selectively install only the modifications that they wanted. Since Oni's seventh anniversary had come and gone, the mod collection was renamed simply the "Anniversary Edition", also known as the "Edition", or the "AE". Released in July 2009, the new AE's framework allowed modders to create their own "[[Making a mod package|packages]]". These could be stored on the {{OMD}}, then downloaded by the player and placed in a certain folder in order to see them in the Installer. Another release was made in May 2010 with more bundled mods and an improved framework.


Minor binary modding had already occurred before Loser's Warehouse (thus, while compiling the [[OBD]] database, ssg had released a version of level0_Final in which all of Konoko's combos were unlocked). Still, it was the Warehouse mod which first showed the "full" potential of binary modding. It inspired a short-lived initiative called the [[OBD:BIP|Binary Improvement Project]], aimed at providing upgraded binaries for all levels, not just level0_Final and level1_Final. BIP was hindered by large upload/download sizes as well as by the potential legal issues involved in distributing entire files from the game, and was promptly put on standby.
Nothing new took place on this front until 2013, by which time the project had fallen into [[User:Iritscen|Iritscen's]] hands due to the retirement of geyser and Gumby. At this time, the release of the new AE included a brand-new Installer written by Alloc which could connect to the Mod Depot, offer all mod packages for download, and update the mods and patches already installed. The AE's bundled content was stripped down to only minimal fixes for the first time, making all notable game modifications into optional third-party mods, and ceasing any attempt at centralized game data improvement.
[[Image:OUP_txmpreplace.jpg|thumb|225px|right|OniUnPacker by [[User:Admin|Alloc]], premiering in fall of 2005, provided a simple interface for finding the resource you wanted and displaying it, as well as replacing it.]]
An equally short-lived alternative to supplying whole levels was developed in the form of [[OUP|OniUnPacker]]-based patches and was termed [[AE:ONK|Oni ni Kanabô]]. The modder would edit his DAT/RAW files in OUP, and then release a single binary resource patch. This was both bandwidth-efficient and more legal-friendly, but suffered from one major limitation: the indexation of DAT resources and the offsets of RAW data vary a lot across the available versions of Oni. Thus, resources exported with OUP were not truly standalone or portable. Also, OUP still lacked the ability to import new resources alongside existing ones.


The advent of Neo's [[OniSplit]] in late 2007 marked a radical improvement. OniSplit breaks Oni's resources into truly standalone and portable modules. These modules can be edited and recombined to create the level data files that Oni reads, but new modules can also be created easily from scratch; image files can be made into texture resources, 3D models into new character modules, etc.
===New content===
Certain game resources were easy enough to reverse-engineer that the community was able to mod them almost from the time of Oni's release, such as textual resources and textures. Some game resources, however, were complex and involved multiple data types, such as character models, and, most obviously, levels themselves. These took a number of years to decipher.


The modified game data contained in .oni files is usually well below any legal standard for copyright violation, but to avoid even distributing these files freely, an alternative is to distribute [[wiktionary:delta|delta]] patches; these patches require the existing Oni data files to be present, then modify that data to get it to the final state, thus not actually giving players any original game content they don't own.
OniSplit first gained the ability to import new character models in April 2008<ref>See [[User_talk:Neo/Archive2|User talk:Neo (Archive 2)]].</ref>. At first, since the focus was on the technical side of things, pre-existing models were used, either by modifying Oni's models, or by finding royalty-free models on web sites and converting them to Oni's 19-bone system. A more dramatic use of the character-importing ability came with the creation of a high-detail Konoko model from scratch in October 2010<ref>[http://oni.bungie.org/forum/viewtopic.php?id=1727 OCF thread, "Konoko HD"].</ref>.
 
A major obstacle with levels was the question of how to generate pathfinding (PF) grids for the AI. Level geometry could be imported fairly easily as of November 2008<ref>See [[User_talk:Neo/Archive5|User talk:Neo (Archive 5)]].</ref>, but AIs could not navigate the level. Manual PF grid creation was first demonstrated with the creation of the Arena level in November 2010<ref>[http://oni.bungie.org/forum/viewtopic.php?id=1736 OCF thread, "Arena Level"].</ref>. Automatic PF grid generation was introduced to OniSplit around June 2011. More comprehensive demonstrations of the community's level creation abilities came with [[User:EdT|EdT]]'s level "Muro's Lair" in August 2011<ref>[http://oni.bungie.org/forum/viewtopic.php?id=2024 OCF thread, "Muro's Lair - Preview"].</ref> and [[User:Samer|Samer]]'s level "Wilderness Preserve" in January 2014<ref>[http://oni.bungie.org/forum/viewtopic.php?id=2698 OCF thread, "NEW LEVEL : Chapter 17: Wilderness Preserve - Hasegawa Adventure"].</ref>.
 
Work continues in all areas of modding Oni, as well as documenting the modding process.
{{clearall}}


OniSplit and delta patches allow the efficient packaging of the comprehensive mod known as the [[Anniversary Edition]], a project currently in beta.
<br clear="all" />
==Engine patching==
==Engine patching==
[[Image:Sfeli Finds Dev Mode.jpg|thumb|225px|right|[[User_talk:SFeLi|SFeLi]] documents his discovery in 2006; in examining the binary, he noticed that the list of cheat codes had a gap in it where one code had been removed; that code was for Developer Access. For more history, see [[Daodan DLL#History|here]].]]
===Finding Dev Mode===
This ultimate form of modding becomes necessary for modders who want to do things the game's code does not currently allow, as well as to fix bugs. Bug fixes were the earliest notable achievement in engine patching, originally performed by a patching application that altered the PC Oni executable directly, but a revolution in this type of modding occurred in 2006 when Developer Mode was finally discovered in retail Oni's code, and a means for unlocking it was needed. The most foolproof method of altering the engine turned out to be a library that Oni loads at runtime. Now known as the [[Daodan DLL]], this library-as-patch can alter many aspects of Oni's behavior. This allows the fixing of bugs (some of which result from new computers running a {{age|2001|1|28}} year old game), as well as the adding of new features, and is a massive help for modders using Windows.  
[[Image:Sfeli Finds Dev Mode.jpg|thumb|225px|right|[[User_talk:SFeLi|SFeLi]] documents his discovery in 2006; in examining the binary, he noticed that the list of cheat codes had a gap in it where one cheat had been deactivated; that code was for Developer Mode. For more history, see [[History of Oni modding/The tale of Dev Mode|The tale of Dev Mode]].]]
 
Engine patching is needed when modders want to do things the game's scripts and data do not currently allow for, as well as to fix bugs. Bug fixes were the earliest notable achievement in engine patching, originally performed by a patching application that altered the Windows Oni executable directly, but a revolution in this type of modding occurred in 2006 when [[Developer Mode]] was finally discovered in retail Oni's code, and a means for unlocking it was needed.
Not long after Daodan DLL was released for Windows, it was discovered that the Mac versions of Oni still had Dev Mode in them, too. Unlike Windows, the Mac version does not seem to have any libraries that can be exploited for runtime patching, so the changes were simply made with a hex editor, directly modifying the executable. In the fall of 2008, it was discovered that numerous BSL functions and variables that were thought to be removed from Oni for the Mac [[Mac beta|before release]] were still present in the code, and the ones that were needed to allow Macs to run (almost) all scripts written for the PC version were also "unlocked" by [[User:EdT|EdT]] using hex edits. This patched executable is freely [http://edt.oni2.net/files/Oni.zip downloadable] in its altered form, which also corrects various age-related bugs and makes some improvements to the game that mirror what the Daodan DLL does for the Windows version.
 
The most foolproof method of altering the engine turned out to be replacing a library that Oni loads at runtime. Originally this was done by placing a library next to Oni.exe called vtuneapi.dll, which contained code written by SFeLi that would replace or patch functions in Oni. This library came to be known as the [[Daodan DLL]]. The code for it was ported from assembly language to C by RossyMiles. Eventually the decision was made to instead replace binkw32.dll (the library for playing Oni's intro and outro movies) because it loaded earlier in Oni's launch sequence. The DLL was later developed by Gumby and then Alloc, fixing more bugs (some of which result from modern computers running a {{Age|2001|1|29}}-year-old game), as well as adding new features. The DLL's features and settings are documented on the page for the DLL linked above.
 
Not long after the Daodan DLL was released for Windows Oni, it was discovered that the Mac versions of Oni still had Dev Mode in them, too. Instead of replacing the Bink Carbon Library that came with Mac Oni, the Omni Group port of the game application to Mac OS X was patched directly with a hex editor. In the fall of 2008, it was discovered that numerous BSL functions and variables that were thought to be removed from Oni for the Mac [[Mac beta 4|before release]] were still present in the code, and the ones which allowed Macs to run (almost) all scripts written for Windows Oni were "unlocked" by EdT using hex edits. Certain changes made by the Daodan DLL for the Windows version were also carried over to the Mac's [[OMNI|PPC game application]] in hex-edit form with Neo's assistance. Later, these changes were made in source for Feral Interactive's [[FERAL|Mac Intel port]].
 
===Bug fixes===
One of the most visible changes to the game that the Daodan DLL (and equivalent hex edits on the Mac side) made possible came about in July '07, when an error in Oni's AI routines was corrected, allowing the AI to dodge incoming fire more consistently. Fans had originally been surprised to see the AI often run blindly into projectiles while being shot at. With the community's patch in place, previously glitched code now works as intended, allowing the AI to dodge projectiles by taking cover.
 
===Multiplayer===
The most ambitious use of engine patching has been to fill in for Oni's missing [[multiplayer]] mode. This is done in Windows through the Daodan DLL in the form of the [[Flatline]] project, and in macOS the [[Zukai]] project operates on a Mac equivalent of the Daodan DLL.
 
===New engine builds===
A new era in Oni's life began with the introduction of [[OniX]], a rebuild of the game engine. It remains to be seen how far this development work will take Oni, but in theory the possibilities are almost endless.


Work continues in all areas of modding Oni.  You can add to the effort by looking at [[Modding brainstorms]] and seeing if you have any skills that can help us out.  Even if you don't know how to make something possible, your ideas are welcome!
<br clear="all" />
----
----
<sup>1</sup>In [[Harry]]'s [http://oni.bungie.org/special/part1/brent.html interview] with Brent Pease, who was the project coordinator at the time, Pease said: "what we will do is, I'm about 90 percent sure of this, is give away the file format." He also promises with equal certainty that the tools for converting 3D Studio Max models into Oni models will be given away. Pease would later leave the project before its completion.  Hardy LeBel, essentially his replacement as Project Lead, would later say that, due to the time crunch, "our primary focus had to be on completing the content and features of the game rather than building in support for the mod community."
<!--I really think general modding tools were promised outright in an early interview, but I can't find it right now. In any case, I'm not sure I want to beat the guys over the head with their own words any more than we already have. --Iritscen-->


<sup>2</sup>Fans wasted no time in dumping the readable text from the executable and trying to figure out how to enable Developer Mode, as [http://carnage.bungie.org/oniresforum/onires.archive.pl?read=35 this thread] from before the game's official release date attests. Little did they know it would take another five years before this particular goal would be reached.
==Footnotes==
<references/>
 
[[Category:Community history]]

Latest revision as of 19:12, 9 November 2023

This article is a historical perspective on Oni modding; for an introduction to how to actually mod Oni, see here. To download mods, you can start here.

As has been documented on the page about Oni's overall development, time ran out for Bungie West to do everything they wanted with this very ambitious game. Just as some features had to be omitted, the promise of modding support[1] also failed to come true. However, even before Oni fans really knew that there would never be official modding tools from Bungie, they were already eagerly diving into the game data to see how they could mod it.[2] Nevertheless, the lack of any documentation or tools for modding undeniably slowed the development of mods within the community. Only since 2010 has the community's modding capabilities covered nearly all aspects of the game.

There are three basic areas in which the game can be modded: scripting, binary resources, and the engine itself. Along the way, we'll look at some key discoveries the community made in more detail.

BSL scripting

Konoko pulls her gun on the secretary in the beginning of Chapter 2, a change to the original cutscene made by a script which tightens up the story and provides a new challenge for veteran players.

Since the .bsl files were plain-text, and in plain sight within the folders that held Oni's game data, it's not surprising that fans quickly learned some basic commands by seeing what Bungie had written, and making their own scripts. Further commands were discovered by dumping readable text from the engine, although not everything worked as it seemed it should.

As scripters were learning BSL, they produced scripts that at first merely modified the original level logic in minor ways. Then they produced more original, often epic fight scripts; later, some ambitious modders scripted totally new objectives and mission layouts (e.g., Mariachi Bear's alternate storyline). Some of the most complex scripts are the OTA scripts which were eventually developed to make up for a lack of multiplayer gameplay. However, fully original scripts are all the more prone to bugs. The first two are easier to set up, but extreme situations tend to overload the engine (as in Blam!) in ways that the scripters have trouble preventing.

Testing the scripts required large numbers of attempts in launching Oni and running the scripts to see how they worked. This process became much faster when the Developer Mode console was unlocked (see below) and individual commands could be tested easily. While this is the easiest way to mod Oni, it's also the most limited; partly because of the lack of full documentation for BSL (which this wiki hopes to provide at some point), and partly because scripts are limited to the resources that an original level has to offer.

Binary resources

Private knowledge, public results

Ian Patterson was probably the most influential early binary hacker, and his trail-blazing program OniTools for the Mac was quickly ported to Windows. It had support for delta patches, and the earliest binary mods were probably based on this method.
OniUnPacker by Alloc, premiering in fall of 2005, provided a simple interface for finding the resource you wanted and displaying it, as well as replacing it.
The mod installer released in 2009. For the first time, players could pick and choose what mods to install.
The Installer released in 2013 could update itself, the player's mods, and the patches made to the game. It was written from scratch in Java for ease of cross-platform support.

Fans with some expertise in parsing files with hex editors found that the level data files held four-letter codes, indicating the start of each resource within that file. Within a week of the street date for Oni, the community had its first resource modding tool, OniTools. While the author did not yet know how all the data worked, his program could at least parse it, and allow the modder to more easily edit each resource without having to count in hexadecimal. It also supported user-friendly replacement of texture maps and allowed in-line viewing of the game's 3D models. There seems to be little evidence that this knowledge was put to use in modding the game resources beyond simple mods such as texture replacements, no doubt due to the lack of a complete understanding of how Oni's dozens of resource types were inter-related.

This did not stop one intrepid programmer named Pierre Terdiman from deciding to import data from Oni into the game engine that he was writing from scratch. Called Konoko Payne, it was a long-term project which took the form of (1) an underlying engine, and (2) a short scenario for it that resembles a sequel to Oni. In order to extract models and animations from Oni and recreate them precisely in his project, Pierre became one of the early pioneers in "reverse engineering" the game's resources, although his acquired knowledge remained largely private, as his work predated the community's organized public attempts at gathering this information.

Hackers unite

Perhaps the first place that knowledge of Oni's data was published online was on Oni Master's site, which started as early as 2003 and was based on the knowledge that had gone into OniTools. Then, in 2005, Ssg opened a site with a much more thorough examination of the game data (the site is now hosted at http://ssg.oni2.net). Later that year, this pool of knowledge was painstakingly moved by geyser and Ssg to its current home on the newly-opened wiki, OniGalore. The purpose of creating the Oni Binary Database was to allow collaboration, and soon Neo appeared on the scene, adding his own knowledge that he had been acquiring independently until then, and collaborating on further investigations into the resource formats. Today at least 95% of the binary resource formats are documented byte for byte thanks to their hard work.

Further reading: History of the Oni community (2005).

Modding by candlelight

Originally, modders had to alter these binary resources manually with hex editors to adapt them to their scripts. Loser was the first modder to manually edit the game data to go along with his scripting, with a massively reshuffled Warehouse. Loser's Warehouse mod included a global component (modified level0_Final), level-specific binary modifications, and a set of customized scripts that took full advantage of the modified binaries.

Minor binary modding had already occurred before Loser's Warehouse (thus, while compiling the OBD database, Ssg had released a version of level0_Final in which all of Konoko's combos were unlocked). Still, it was the Warehouse mod released in 2005 which first showed the "full" potential of binary modding. It inspired a short-lived initiative called the Binary Improvement Project, aimed at providing upgraded binaries for all levels, not just level0_Final and level1_Final.

BIP was hindered by large upload/download sizes as well as by the potential legal issues involved in distributing entire files from the game; in 2006 it was replaced by a smarter alternative to supplying whole levels, in the form of OniUnPacker-based patches and was termed Oni ni Kanabô. The modder would edit his .dat/.raw files in OUP, and then release a single binary resource patch. This was both bandwidth-efficient and more legal-friendly, but suffered from one major limitation: the indexation and offsets of data vary a lot across the available versions of Oni. Thus, resources exported with OUP were not truly standalone or portable. Also, OUP still lacked the ability to import new resources alongside existing ones.

Modular modding

The advent of Neo's OniSplit in late 2007 marked a radical improvement to our modding methods. A command-line tool that runs on Windows and Macs, OniSplit breaks Oni's resources into standalone files, making mods truly portable across different retail versions of Oni. These resources can be exported to standard formats as text, images, sounds and 3D models, edited, and recombined into the level data files that Oni expects, and new resources can also be created from scratch. To take advantage of this revolution in modding, geyser began preparing an OniSplit-based port of some of the work done on BIP and ONK, which he hoped would be ready for the seventh anniversary of Oni's release in January 2008.

This "Seventh Anniversary Edition" had major releases in January and July 2008. The tradition continued of using a central modding project to organize a general revision of Oni's content, as seen on pages like AE:Ninja. The project then changed hands to Gumby, who designed a framework and a GUI-based installer that allowed players to selectively install only the modifications that they wanted. Since Oni's seventh anniversary had come and gone, the mod collection was renamed simply the "Anniversary Edition", also known as the "Edition", or the "AE". Released in July 2009, the new AE's framework allowed modders to create their own "packages". These could be stored on the Oni Mod Depot, then downloaded by the player and placed in a certain folder in order to see them in the Installer. Another release was made in May 2010 with more bundled mods and an improved framework.

Nothing new took place on this front until 2013, by which time the project had fallen into Iritscen's hands due to the retirement of geyser and Gumby. At this time, the release of the new AE included a brand-new Installer written by Alloc which could connect to the Mod Depot, offer all mod packages for download, and update the mods and patches already installed. The AE's bundled content was stripped down to only minimal fixes for the first time, making all notable game modifications into optional third-party mods, and ceasing any attempt at centralized game data improvement.

New content

Certain game resources were easy enough to reverse-engineer that the community was able to mod them almost from the time of Oni's release, such as textual resources and textures. Some game resources, however, were complex and involved multiple data types, such as character models, and, most obviously, levels themselves. These took a number of years to decipher.

OniSplit first gained the ability to import new character models in April 2008[3]. At first, since the focus was on the technical side of things, pre-existing models were used, either by modifying Oni's models, or by finding royalty-free models on web sites and converting them to Oni's 19-bone system. A more dramatic use of the character-importing ability came with the creation of a high-detail Konoko model from scratch in October 2010[4].

A major obstacle with levels was the question of how to generate pathfinding (PF) grids for the AI. Level geometry could be imported fairly easily as of November 2008[5], but AIs could not navigate the level. Manual PF grid creation was first demonstrated with the creation of the Arena level in November 2010[6]. Automatic PF grid generation was introduced to OniSplit around June 2011. More comprehensive demonstrations of the community's level creation abilities came with EdT's level "Muro's Lair" in August 2011[7] and Samer's level "Wilderness Preserve" in January 2014[8].

Work continues in all areas of modding Oni, as well as documenting the modding process.

Engine patching

Finding Dev Mode

SFeLi documents his discovery in 2006; in examining the binary, he noticed that the list of cheat codes had a gap in it where one cheat had been deactivated; that code was for Developer Mode. For more history, see The tale of Dev Mode.

Engine patching is needed when modders want to do things the game's scripts and data do not currently allow for, as well as to fix bugs. Bug fixes were the earliest notable achievement in engine patching, originally performed by a patching application that altered the Windows Oni executable directly, but a revolution in this type of modding occurred in 2006 when Developer Mode was finally discovered in retail Oni's code, and a means for unlocking it was needed.

The most foolproof method of altering the engine turned out to be replacing a library that Oni loads at runtime. Originally this was done by placing a library next to Oni.exe called vtuneapi.dll, which contained code written by SFeLi that would replace or patch functions in Oni. This library came to be known as the Daodan DLL. The code for it was ported from assembly language to C by RossyMiles. Eventually the decision was made to instead replace binkw32.dll (the library for playing Oni's intro and outro movies) because it loaded earlier in Oni's launch sequence. The DLL was later developed by Gumby and then Alloc, fixing more bugs (some of which result from modern computers running a 23-year-old game), as well as adding new features. The DLL's features and settings are documented on the page for the DLL linked above.

Not long after the Daodan DLL was released for Windows Oni, it was discovered that the Mac versions of Oni still had Dev Mode in them, too. Instead of replacing the Bink Carbon Library that came with Mac Oni, the Omni Group port of the game application to Mac OS X was patched directly with a hex editor. In the fall of 2008, it was discovered that numerous BSL functions and variables that were thought to be removed from Oni for the Mac before release were still present in the code, and the ones which allowed Macs to run (almost) all scripts written for Windows Oni were "unlocked" by EdT using hex edits. Certain changes made by the Daodan DLL for the Windows version were also carried over to the Mac's PPC game application in hex-edit form with Neo's assistance. Later, these changes were made in source for Feral Interactive's Mac Intel port.

Bug fixes

One of the most visible changes to the game that the Daodan DLL (and equivalent hex edits on the Mac side) made possible came about in July '07, when an error in Oni's AI routines was corrected, allowing the AI to dodge incoming fire more consistently. Fans had originally been surprised to see the AI often run blindly into projectiles while being shot at. With the community's patch in place, previously glitched code now works as intended, allowing the AI to dodge projectiles by taking cover.

Multiplayer

The most ambitious use of engine patching has been to fill in for Oni's missing multiplayer mode. This is done in Windows through the Daodan DLL in the form of the Flatline project, and in macOS the Zukai project operates on a Mac equivalent of the Daodan DLL.

New engine builds

A new era in Oni's life began with the introduction of OniX, a rebuild of the game engine. It remains to be seen how far this development work will take Oni, but in theory the possibilities are almost endless.


Footnotes

  1. In Harry's interview, Brent Pease, the Project Lead, said: "What we will do is, I'm about 90 percent sure of this, is give away the file format." He also promises with equal certainty that the tools for converting 3D Studio Max models into Oni models will be given away. Pease eventually left the project before its completion. Hardy LeBel, his replacement, would later say, "[Due to the time crunch] our primary focus had to be on completing the content and features of the game rather than building in support for the mod community." Additionally, Bungie no longer owned the game by the time it shipped, so it would have been legally questionable to release header files from the source code, which would be the simplest and quickest way to "give away the file format".
  2. Fans wasted no time in dumping the readable text from the executable and trying to figure out how to enable Developer Mode, starting before Oni's official release date by analyzing the demo. Little did they know it would take another five years before this particular goal would be reached.
  3. See User talk:Neo (Archive 2).
  4. OCF thread, "Konoko HD".
  5. See User talk:Neo (Archive 5).
  6. OCF thread, "Arena Level".
  7. OCF thread, "Muro's Lair - Preview".
  8. OCF thread, "NEW LEVEL : Chapter 17: Wilderness Preserve - Hasegawa Adventure".