Anniversary Edition/Framework: Difference between revisions

From OniGalore
Jump to navigation Jump to search
(partial rewrite, spelling out more clearly that enviro TXMPs are *not* globalized)
(rewriting Installation to include the section on XML patching; hopefully this is clearer in list format)
Line 1: Line 1:
The Anniversary Edition Seven introduced a revised framework which installs into the directory Oni/AE/, separate from any previous Editions, which installed into Oni/Edition/. Following is a rundown of what the Installer is doing, from start to finish. To learn how to make your own mod in the package format used by the AE Installer, see [[Making a mod package]]. To understand the material below, you should first be familiar with Oni's data files on a general level, so [[OBD]] is recommended reading.
The Anniversary Edition Seven introduced a revised framework which installs into the directory Oni/AE/, separate from any previous Editions, which installed into Oni/Edition/. Following is a technical description of how mods are installed, from start to finish. To learn how to make your own mod in the package format used by the AE Installer, see [[Making a mod package]]. To understand the material below, you should first be familiar with Oni's data files on a general level, so [[OBD]] is recommended reading.


==Initialization==
==Initialization==
When first running the Installer, you will be required to initialize the AE. This is a one-time process that uses [[OniSplit]] to break the .dat/.raw(/.sep) files in Oni's [[GDF]] into their constituent resources, yielding thousands of .oni files. These files are stored in temporary folders, one for each level. Next, the Installer performs what is called globalization. When developing Oni, in order to save memory, [[BWest]] put in each level's data files only the characters, textures, sounds, etc. that were needed. Since we have far more RAM these days, we can make this data available in all levels in order to allow more versatile modding. To do this, the Installer moves many of the .onis that it exported to each specific level's folder into the folder for level 0, discarding duplicates in the process (since many of the same resources were duplicated throughout the original levels' data).
When first running the Installer, the user is required to initialize the AE. This is a one-time process that uses [[OniSplit]] to break the .dat/.raw(/.sep) files in Oni's [[GDF]] into their constituent resources, yielding thousands of .oni files. These files are stored in temporary folders, one for each level.


There is a major exception to this globalization, however. Each level's [[AKEV]] and all child resources (e.g. all environmental textures), as well as the levels' title, fail and win screens, and [[LSI]], are <u>left</u> in that level's folder. Not only would globalized environmental textures gobble up memory unnecessarily, but if a modder replaced an environmental TXMP that had been globalized into level 0, it would show up in all levels, which can have undesirable aesthetic side effects. So, what data <u>is</u> globalized? Primarily, all characters, their animations (even the ones for specific cutscenes), all sounds, all particles, and any unused textures that were not connected to AKEVs.
Next, the Installer performs what is called globalization. When developing Oni, in order to save memory, [[BWest]] put in each level's data files only the characters, textures, sounds, etc. that were needed. Since we have far more RAM these days, we can make this data available in all levels in order to allow more versatile modding. To do this, the Installer moves many of the .onis from each specific level's folder into level 0's folder, discarding duplicates in the process (since many of the same resources were duplicated throughout the original levels' data).


Finally, the Installer recombines those re-shuffled folders of .oni files into the .dat/.raw(/.sep) files for each level, saving them in AE/AEInstaller/vanilla/. Note that no data has been altered yet, just the resources reorganized.
There is a major exception to this globalization, however. Each level's [[AKEV]] and all child resources (e.g. all environmental textures), as well as the levels' title, fail and win screens, and [[LSI]], are <u>left</u> in that level's folder. Not only would globalized textures gobble up memory unnecessarily, but if a modder replaced an environmental TXMP that had been globalized into level 0, it would show up in all levels, which could have undesirable aesthetic side effects. So, what data <u>is</u> globalized? Primarily, all characters, their animations (even the ones for specific cutscenes), all sounds, all particles, and any unused textures that were not connected to AKEVs.
 
Finally, the Installer builds new .dat/.raw(/.sep) files for each level from those re-shuffled folders of .oni files, saving them in AE/AEInstaller/vanilla/. Note that no data has been <u>altered</u> yet, just the resources reorganized.


==Installation==
==Installation==
After this, the user can choose some mods to install. When the Install button is clicked, the Installer starts off by deleting the existing .dat/.raw(/.sep) files in AE/GDF to start from a blank slate. It then iterates through each mod package you selected, and sends a command to OniSplit to combine the .dat for that level (in AEInstaller/vanilla/) with the .oni files from whichever packages have files meant for that level. The final .dat/.raw(/.sep) files are then created in AE/GDF.
After this, the user can choose some mods to install. When the Install button is clicked, the Installer (AEI) performs the following actions:
 
1. Look at which levels will be altered by current selection of mods, and delete those levels' .dat/.raw(/.sep) files from AE's GDF.
 
2. Collect .oni files that will be patched by an XML patch mod by calling OniSplit with -export: switch. The .oni files are stored in a system temporary directory, so they will be automatically deleted by the OS later.
 
The Installer first looks for any patches (.oni-patch files) among the user's chosen mods. An .oni-patch is named according to the resource it modifies, e.g. TRAMKONCOMbk_fw_kick.oni-patch, and placed in a folder with a level's name, such as level2_Final/, in order to indicate which level it patches (just like the .oni files in mod packages).
 
Because patches are intended to take precedence over any regular mods, the Installer checks whether any selected mods have a .oni file, meant for the same level, which matches the .oni-patch's name. If so, it converts the mod's .oni file to XML; otherwise it extracts the resource by that name from the vanilla .dat of the appropriate level (stored in AE/AEInstaller/vanilla/). If more than one mod has a matching .oni, the highest-numbered mod's .oni is used.
 
3. Convert .oni files from step 2 to XML by calling OniSplit with -extract:xml switch. The XML files will be deleted after installation is done.
 
4. Process XML patches by calling XmlTools to operate on XML files from step 3.
 
The AEI makes a single temporary patch file which lists inside of it a series of commands to XmlTools to perform the instructions in each individual .oni-patch file. This "master patch" is made for performance reasons and should not affect the modder who creates .oni-patches. Read the documentation for [[XmlTools]] to understand how the patch files are actually processed, and read [[Making a patch mod]] to learn how to make an XML patch mod.
 
5. Convert patched XML files from step 4 back to .oni files by calling OniSplit with -create switch.


When one package overlaps with another in some way, such as two packages that provide new textures for the same character, which one will take precedence? The answer is that the Installer goes in ascending order by package number, so if five copies of the same .oni file are encountered one after another in five packages, only the last-encountered one will be used. Thus, a package numbered 11111 will override package 11110 wherever their files conflict, when they are both installed. Note that this is on a file-by-file basis, so package 11111 could be used intentionally to override only certain files in 11110, say, to improve or fix them. On the other hand, this might be an undesirable interaction. It's the responsibility of the modders to be aware when there's an unwanted interaction and to mark their mod as being incompatible with the other one.
The newly-made .oni files, with the patches applied, overwrite the .oni files in the directory in which they were placed in step 2.


==XML patching==
6. Build new level files by calling OniSplit with "pack -out" switch.
To read about the purpose of XML patches, visit the page [[Making a patch mod]]. Here, we'll simply describe the process by which they are installed. XML patching takes place at the beginning of each mod installation, because the .oni files need to be patched before they are combined back into .dats. Therefore, when about to begin installing the user's selected mods, the Installer looks for any patches (.oni-patch files) in those mods. The .oni-patches are named according to the file they modify, e.g. TRAMKONCOMbk_fw_kick.oni-patch, and placed in numbered folders such as level2_Final/, just like mods with .oni files.


For each file that is to be patched, the AEI looks for the highest-numbered package which has such a .oni file in the same level number; in other words, patches go on top of all .oni mods, thus they have higher precedence than any package does by number -- regardless of the number of the package containing said patch. If there is no copy of that .oni file in any selected mod, the AEI extracts the .oni from the vanilla .dat of the appropriate level (stored in AE/AEInstaller/vanilla/).
For each level, the AEI passes OniSplit the location of the vanilla AE .dat (in AE/AEInstaller/vanilla/) plus a list of directories that contain .oni files meant for that level, in ascending order by mod number so that higher-numbered mods override lower ones*. The last directory in the list is the temporary directory containing the patched .oni files from step 5, so that the patched files override any other versions of those resources. The final .dat/.raw(/.sep) files have now been created in the AE's GDF.


Once all the .oni files to be patched have been collected from either the other mods or vanilla .dats, the AEI makes a single temporary patch file which lists inside of it a series of commands to XmlTools to run each individual patch file on the .oni file named by that .oni-patch. This "master patch" is made for performance reasons and should not affect the modder who creates .oni-patches. Read the documentation for [[XmlTools]] to understand how the patch files are actually processed. The .oni files having been patched, their containing folder is then combined, along with the folders of unpatched .onis from the other selected mods, into .dats, as described in "Installation".
<nowiki>*</nowiki>Note that this is on a file-by-file basis. So a package numbered 11111 will only override package 11110 wherever their files conflict, when they are both installed. Thus a package numbered 11111 could be used intentionally to override only certain files in 11110, e.g., to improve or fix them. On the other hand, a partial override by a higher package might be an undesirable interaction. It's the responsibility of the modders to be aware when there's an unwanted interaction and to mark their mod as being incompatible with the other one using the Incompatibility flag described in [[Making a mod package]].


[[Category:Anniversary Edition]]
[[Category:Anniversary Edition]]

Revision as of 22:43, 28 June 2015

The Anniversary Edition Seven introduced a revised framework which installs into the directory Oni/AE/, separate from any previous Editions, which installed into Oni/Edition/. Following is a technical description of how mods are installed, from start to finish. To learn how to make your own mod in the package format used by the AE Installer, see Making a mod package. To understand the material below, you should first be familiar with Oni's data files on a general level, so OBD is recommended reading.

Initialization

When first running the Installer, the user is required to initialize the AE. This is a one-time process that uses OniSplit to break the .dat/.raw(/.sep) files in Oni's GDF into their constituent resources, yielding thousands of .oni files. These files are stored in temporary folders, one for each level.

Next, the Installer performs what is called globalization. When developing Oni, in order to save memory, BWest put in each level's data files only the characters, textures, sounds, etc. that were needed. Since we have far more RAM these days, we can make this data available in all levels in order to allow more versatile modding. To do this, the Installer moves many of the .onis from each specific level's folder into level 0's folder, discarding duplicates in the process (since many of the same resources were duplicated throughout the original levels' data).

There is a major exception to this globalization, however. Each level's AKEV and all child resources (e.g. all environmental textures), as well as the levels' title, fail and win screens, and LSI, are left in that level's folder. Not only would globalized textures gobble up memory unnecessarily, but if a modder replaced an environmental TXMP that had been globalized into level 0, it would show up in all levels, which could have undesirable aesthetic side effects. So, what data is globalized? Primarily, all characters, their animations (even the ones for specific cutscenes), all sounds, all particles, and any unused textures that were not connected to AKEVs.

Finally, the Installer builds new .dat/.raw(/.sep) files for each level from those re-shuffled folders of .oni files, saving them in AE/AEInstaller/vanilla/. Note that no data has been altered yet, just the resources reorganized.

Installation

After this, the user can choose some mods to install. When the Install button is clicked, the Installer (AEI) performs the following actions:

1. Look at which levels will be altered by current selection of mods, and delete those levels' .dat/.raw(/.sep) files from AE's GDF.

2. Collect .oni files that will be patched by an XML patch mod by calling OniSplit with -export: switch. The .oni files are stored in a system temporary directory, so they will be automatically deleted by the OS later.

The Installer first looks for any patches (.oni-patch files) among the user's chosen mods. An .oni-patch is named according to the resource it modifies, e.g. TRAMKONCOMbk_fw_kick.oni-patch, and placed in a folder with a level's name, such as level2_Final/, in order to indicate which level it patches (just like the .oni files in mod packages).

Because patches are intended to take precedence over any regular mods, the Installer checks whether any selected mods have a .oni file, meant for the same level, which matches the .oni-patch's name. If so, it converts the mod's .oni file to XML; otherwise it extracts the resource by that name from the vanilla .dat of the appropriate level (stored in AE/AEInstaller/vanilla/). If more than one mod has a matching .oni, the highest-numbered mod's .oni is used.

3. Convert .oni files from step 2 to XML by calling OniSplit with -extract:xml switch. The XML files will be deleted after installation is done.

4. Process XML patches by calling XmlTools to operate on XML files from step 3.

The AEI makes a single temporary patch file which lists inside of it a series of commands to XmlTools to perform the instructions in each individual .oni-patch file. This "master patch" is made for performance reasons and should not affect the modder who creates .oni-patches. Read the documentation for XmlTools to understand how the patch files are actually processed, and read Making a patch mod to learn how to make an XML patch mod.

5. Convert patched XML files from step 4 back to .oni files by calling OniSplit with -create switch.

The newly-made .oni files, with the patches applied, overwrite the .oni files in the directory in which they were placed in step 2.

6. Build new level files by calling OniSplit with "pack -out" switch.

For each level, the AEI passes OniSplit the location of the vanilla AE .dat (in AE/AEInstaller/vanilla/) plus a list of directories that contain .oni files meant for that level, in ascending order by mod number so that higher-numbered mods override lower ones*. The last directory in the list is the temporary directory containing the patched .oni files from step 5, so that the patched files override any other versions of those resources. The final .dat/.raw(/.sep) files have now been created in the AE's GDF.

*Note that this is on a file-by-file basis. So a package numbered 11111 will only override package 11110 wherever their files conflict, when they are both installed. Thus a package numbered 11111 could be used intentionally to override only certain files in 11110, e.g., to improve or fix them. On the other hand, a partial override by a higher package might be an undesirable interaction. It's the responsibility of the modders to be aware when there's an unwanted interaction and to mark their mod as being incompatible with the other one using the Incompatibility flag described in Making a mod package.