Anniversary Edition/Framework: Difference between revisions

From OniGalore
Jump to navigation Jump to search
(→‎Package numbering: we found a use for 7xxxx)
(→‎Initialization: correction on globalization)
 
(16 intermediate revisions by 2 users not shown)
Line 1: Line 1:
The 2009-07 release of the Edition introduced a revised framework. You should place the unzipped contents of the AE release for your platform into a fresh copy of Oni's folder, rather than a copy of Oni that already had an older Edition installed. Following is a rundown of how globalization and mod installation work. To learn how to make your own mod package, which is required in order for the AE Installer to handle your mod properly, see [[Making a mod package]].
The Anniversary Edition Seven introduced a revised framework which installs into the directory Oni/AE/, separate from any previous Editions, which installed into Oni/Edition/. Following is a technical description of how mods are installed, from start to finish. To learn how to make your own mod in the package format used by the AE Installer, see [[Making a mod package]]. To understand the material below, you should first be familiar with Oni's data files on a general level, so [[OBD]] is recommended reading.


==The globalization process==
==Initialization==
When first running the Installer, this is a one-time process that takes the vanilla .dats in Oni's GDF and breaks them into thousands of .oni files in folders like Edition/GDF/level1_Final. Next, the Installer combines those folders of .oni files into singleton, large .oni files, one for each level, in Edition/install/VanillaDats/. Note that no data has been modded yet, just reformatted. This process of pre-compiling the data results in a large speed increase over previous recombination methods.
When first running the Installer, the user is required to initialize the AE. This is a one-time process that uses [[OniSplit]] to break the .dat/.raw(/.sep) files in Oni's [[GDF]] into their constituent resources, yielding thousands of .oni files. These files are stored in temporary folders, one for each level.


==The mod installation process==
Next, the Installer performs what is called globalization. When developing Oni, in order to save memory, [[BWest]] put in each level's data files only the characters, textures, sounds, etc. that were needed. Since we have far more RAM these days, we can make this data available in all levels in order to allow more versatile modding. To do this, the Installer moves many of the .onis from each specific level's folder into level 0's folder, discarding duplicates in the process (since many of the same resources were duplicated throughout the original levels' data).
Right after this, the Installer prompts you to choose some mods to install. When you hit the Install button, the Installer starts off by deleting the existing .dat/.raw(/.sep) files in Edition/GDF to start from a blank slate. It then iterates through each mod package you selected, and sends a command to OniSplit to combine the singleton .oni for that level (in VanillaDats) with the .oni files in whichever packages have files meant for that level. The final .dat/.raw(/.sep) files are then created in Edition/GDF.


==Implications of the new process==
There is a major exception to this globalization, however. Each level's [[AKEV]] and all child resources (e.g. all environmental textures), as well as the level's title, fail and win screens, and [[LSI]], are <u>left</u> in that level's folder. If the textures were globalized and a modder intended to replace a texture on just one level, it would show up in all levels, which might not be the desired effect. So, what data <u>is</u> globalized? Primarily, all characters, their animations (even the ones for specific cutscenes), all sounds, and any textures that are not connected to AKEVs (e.g., unused textures). All particles are already globalized in vanilla Oni, and these remain globalized in the AE.
The import of this is that the folders of .oni files in Edition/GDF/ ''are never used again'' after the globalization is done. In the old way of doing things, these folders were what you modded files in, and then recombined to generate new .dat/.raw(/.sep) files. However, if you tried that approach now, and altered those folders' files, and recompiled them, you would find yourself playing an Oni with only your mod and no AE mods, because the <tt>-import</tt> command you used on the "_Final" folder left out the AE's mods in Edition/install/packages/. And the next time you installed or removed a mod using the Installer, you would find yourself playing an Oni with only the AE mods, and not your own. Your modded files would still be in the "_Final" folder you placed them in, but the <tt>-import</tt> command that the Installer used didn't look at those "_Final" folders, it worked from the giant singleton .oni files for each level that were in VanillaDats instead.


Put even more simply, your new modded files have a mutual exclusion with the AE's modded files; only one or the other can be active at a time. No actual harm is being done in terms of lost data if you attempt to mod in this old way, but it means that the smart way to mod is always to make a package and install it through the Installer, even if you are merely testing something. This has the nice side effect of forcing you to be organized :-)
Finally, the Installer builds new .dat/.raw(/.sep) files for each level from those re-shuffled folders of .oni files, saving them in AE/AEInstaller/vanilla/. Note that no data has been <u>altered</u> yet, just the resources reorganized.


==Package numbering==
==Installation==
When the Installer sees that one package overlaps with another in some way, such as two packages that provide new textures for the same character, which one will take precedence? The answer is that the Installer goes by the number at the front of the name of the package folder. A package numbered 11111 will override package 11110 when they are installed into Oni.
After this, the user can choose some mods to install. When the Install button is clicked, the Installer (AEI) performs the following actions:


So, how do you pick a number for your mod? Here's how you pick the first of the five digits:
1. Look at which levels will be altered by comparing the current selection of mods to the set of previously installed mods, and delete those levels' .dat/.raw(/.sep) files from AE's GDF.


'''10000s - Weapons'''
2. Collect .oni files that will be patched by an XML patch mod by calling OniSplit with -export: switch. The .oni files are stored in a system temporary directory, so they will be automatically deleted by the OS later.


'''20000s - Characters (and character textures)'''
:The Installer first looks for any patches (.oni-patch files) among the user's chosen mods. An .oni-patch is named according to the resource it modifies, e.g. TRAMKONCOMbk_fw_kick.oni-patch, and placed in a folder with a level's name, such as level2_Final/, in order to indicate which level it patches (just like the .oni files in mod packages).


'''30000s - Particles'''
:Because patches are intended to take precedence over any regular mods, the Installer checks whether any selected mods have a .oni file, meant for the same level, which matches the .oni-patch's name. If so, it converts the mod's .oni file to XML; otherwise it extracts the resource by that name from the vanilla .dat of the appropriate level (stored in AE/AEInstaller/vanilla/). If more than one mod has a matching .oni, the highest-numbered mod's .oni is used.


'''40000s - Animations (non-combat)'''
3. Convert .oni files from step 2 to XML by calling OniSplit with -extract:xml switch. The XML files will be deleted after installation is done.


'''50000s - Combat animations (incl. animation files)'''
4. Process XML patches by calling XmlTools to operate on XML files from step 3.


'''60000s - Sounds'''
:The AEI makes a single temporary patch file which lists inside of it a series of commands to XmlTools to perform the instructions in each individual .oni-patch file. This "master patch" is made for performance reasons and should not affect the modder who creates .oni-patches. Read the documentation for [[XmlTools]] to understand how the patch files are actually processed, and read [[Making a patch mod]] to learn how to make an XML patch mod.


'''70000s - UI (e.g., new menu screens)'''
5. Convert patched XML files from step 4 back to .oni files by calling OniSplit with -create switch.


'''80000s - Levels (and level textures)'''
:The newly-made .oni files, with the patches applied, overwrite the .oni files in the directory in which they were placed in step 2.


'''90000s - Scripts (BSL)'''
6. Build new level files by calling OniSplit with "pack -out" switch.


Once you know which digit to start with, you should use the remaining digits to number your mod in a way that fits in with the other mod packages. For example, here's some hypothetical package numbers and what they tell you about the package:
:For each level, the AEI passes OniSplit the location of the vanilla AE .dat (in AE/AEInstaller/vanilla/) plus a list of directories that contain .oni files meant for that level in ascending order by mod number, so that higher-numbered mods override lower ones*. The last directory in the list is the temporary directory containing the patched .oni files from step 5, so that the patched files override any other versions of those resources. The final .dat/.raw(/.sep) files have now been created in the AE's GDF.


41000 (a mod that tweaks animations)
::<nowiki>*</nowiki>Note that this is on a file-by-file basis. So a package numbered 11111 will only override package 11110 wherever their files conflict, when they are both installed. Thus a package numbered 11111 could be used intentionally to override only certain files in 11110, e.g., to improve or fix them. On the other hand, a partial override by a higher package might be an undesirable interaction. It's the responsibility of the modders to be aware when there's an unwanted interaction and to mark their mod as being incompatible with the other one using the Incompatibility flag described in [[Making a mod package]].


41100 (this mod might be related to mod 41000, or else you are just looking for another free number and 42-4900 are taken)
7. BSL files are treated similarly to .oni files. Mods with bsl/ folders are considered in ascending order and the highest-numbered mod that affects a given level has its BSL files copied to the AE IGMD folder. The special case with BSL is the "addon" feature. When a mod's Mod_Info.cfg uses the "HasBSL->Addon" flag, those mods' BSL files are considered after the highest-numbered regular BSL mod (or vanilla BSL, in the absence of any mod) is installed for each level. The addon BSL files are copied into the desired level folders, overwriting whatever file by the same name already exists there.


41010 (probably someone's addition or improvement to mod 41000)
:Secret feature: If a modder is experimenting with his own script in a given level's BSL folder, he can place an ignore.txt file in that folder (no contents necessary) in order to prevent the AEI from wiping out the BSL in that folder in case a mod tries to affect it.


41001 (must be an addition or improvement for mod 41000, or why else would they be numbered so closely?)
8. Plain files (files in a mod's plain/ directory) are copied into the directory specified within the platform folders within the plain/ folder. E.g., if the mod contains the directory plain/common/Tools/, then the files in that directory will be copied into a Tools folder in the AE/ directory.


Okay, so now you get the idea, but how do you know which numbers are free? First, check the numbers in use by the AE, which are listed on the [[Anniversary Edition/Packages|AE Packages]] page. Next, go to the [http://mods.oni2.net Mod Depot], and choose "By Install Method>Package" from the sidebar to see all packaged mods. The names of the files (under the "Download Link" column) will tell you the numbers for these third-party packages. Any number not used by an AE package or a package on the Depot should be safe to use.
9. If the user's AEI preferences call for it, the intro and/or outro videos are copied from the vanilla Oni GDF to the AE GDF.
 
10. Any new levels that were added by mods to the game are unlocked by editing persist.dat so that they are accessible from the Load Game dialog.


[[Category:Anniversary Edition]]
[[Category:Anniversary Edition]]

Latest revision as of 20:33, 10 June 2020

The Anniversary Edition Seven introduced a revised framework which installs into the directory Oni/AE/, separate from any previous Editions, which installed into Oni/Edition/. Following is a technical description of how mods are installed, from start to finish. To learn how to make your own mod in the package format used by the AE Installer, see Making a mod package. To understand the material below, you should first be familiar with Oni's data files on a general level, so OBD is recommended reading.

Initialization

When first running the Installer, the user is required to initialize the AE. This is a one-time process that uses OniSplit to break the .dat/.raw(/.sep) files in Oni's GDF into their constituent resources, yielding thousands of .oni files. These files are stored in temporary folders, one for each level.

Next, the Installer performs what is called globalization. When developing Oni, in order to save memory, BWest put in each level's data files only the characters, textures, sounds, etc. that were needed. Since we have far more RAM these days, we can make this data available in all levels in order to allow more versatile modding. To do this, the Installer moves many of the .onis from each specific level's folder into level 0's folder, discarding duplicates in the process (since many of the same resources were duplicated throughout the original levels' data).

There is a major exception to this globalization, however. Each level's AKEV and all child resources (e.g. all environmental textures), as well as the level's title, fail and win screens, and LSI, are left in that level's folder. If the textures were globalized and a modder intended to replace a texture on just one level, it would show up in all levels, which might not be the desired effect. So, what data is globalized? Primarily, all characters, their animations (even the ones for specific cutscenes), all sounds, and any textures that are not connected to AKEVs (e.g., unused textures). All particles are already globalized in vanilla Oni, and these remain globalized in the AE.

Finally, the Installer builds new .dat/.raw(/.sep) files for each level from those re-shuffled folders of .oni files, saving them in AE/AEInstaller/vanilla/. Note that no data has been altered yet, just the resources reorganized.

Installation

After this, the user can choose some mods to install. When the Install button is clicked, the Installer (AEI) performs the following actions:

1. Look at which levels will be altered by comparing the current selection of mods to the set of previously installed mods, and delete those levels' .dat/.raw(/.sep) files from AE's GDF.

2. Collect .oni files that will be patched by an XML patch mod by calling OniSplit with -export: switch. The .oni files are stored in a system temporary directory, so they will be automatically deleted by the OS later.

The Installer first looks for any patches (.oni-patch files) among the user's chosen mods. An .oni-patch is named according to the resource it modifies, e.g. TRAMKONCOMbk_fw_kick.oni-patch, and placed in a folder with a level's name, such as level2_Final/, in order to indicate which level it patches (just like the .oni files in mod packages).
Because patches are intended to take precedence over any regular mods, the Installer checks whether any selected mods have a .oni file, meant for the same level, which matches the .oni-patch's name. If so, it converts the mod's .oni file to XML; otherwise it extracts the resource by that name from the vanilla .dat of the appropriate level (stored in AE/AEInstaller/vanilla/). If more than one mod has a matching .oni, the highest-numbered mod's .oni is used.

3. Convert .oni files from step 2 to XML by calling OniSplit with -extract:xml switch. The XML files will be deleted after installation is done.

4. Process XML patches by calling XmlTools to operate on XML files from step 3.

The AEI makes a single temporary patch file which lists inside of it a series of commands to XmlTools to perform the instructions in each individual .oni-patch file. This "master patch" is made for performance reasons and should not affect the modder who creates .oni-patches. Read the documentation for XmlTools to understand how the patch files are actually processed, and read Making a patch mod to learn how to make an XML patch mod.

5. Convert patched XML files from step 4 back to .oni files by calling OniSplit with -create switch.

The newly-made .oni files, with the patches applied, overwrite the .oni files in the directory in which they were placed in step 2.

6. Build new level files by calling OniSplit with "pack -out" switch.

For each level, the AEI passes OniSplit the location of the vanilla AE .dat (in AE/AEInstaller/vanilla/) plus a list of directories that contain .oni files meant for that level in ascending order by mod number, so that higher-numbered mods override lower ones*. The last directory in the list is the temporary directory containing the patched .oni files from step 5, so that the patched files override any other versions of those resources. The final .dat/.raw(/.sep) files have now been created in the AE's GDF.
*Note that this is on a file-by-file basis. So a package numbered 11111 will only override package 11110 wherever their files conflict, when they are both installed. Thus a package numbered 11111 could be used intentionally to override only certain files in 11110, e.g., to improve or fix them. On the other hand, a partial override by a higher package might be an undesirable interaction. It's the responsibility of the modders to be aware when there's an unwanted interaction and to mark their mod as being incompatible with the other one using the Incompatibility flag described in Making a mod package.

7. BSL files are treated similarly to .oni files. Mods with bsl/ folders are considered in ascending order and the highest-numbered mod that affects a given level has its BSL files copied to the AE IGMD folder. The special case with BSL is the "addon" feature. When a mod's Mod_Info.cfg uses the "HasBSL->Addon" flag, those mods' BSL files are considered after the highest-numbered regular BSL mod (or vanilla BSL, in the absence of any mod) is installed for each level. The addon BSL files are copied into the desired level folders, overwriting whatever file by the same name already exists there.

Secret feature: If a modder is experimenting with his own script in a given level's BSL folder, he can place an ignore.txt file in that folder (no contents necessary) in order to prevent the AEI from wiping out the BSL in that folder in case a mod tries to affect it.

8. Plain files (files in a mod's plain/ directory) are copied into the directory specified within the platform folders within the plain/ folder. E.g., if the mod contains the directory plain/common/Tools/, then the files in that directory will be copied into a Tools folder in the AE/ directory.

9. If the user's AEI preferences call for it, the intro and/or outro videos are copied from the vanilla Oni GDF to the AE GDF.

10. Any new levels that were added by mods to the game are unlocked by editing persist.dat so that they are accessible from the Load Game dialog.