20,365
edits
OBD:Instance file format: Difference between revisions
m
clarified and corrected OniSplit checksum note that has been confusing me – still not sure that all the listed types are stored in 1.1 format since the statement about TXMP was incorrect (unless this was written and true for OS 1.0?)
(moved engine version section to new page Engine terminology) |
m (clarified and corrected OniSplit checksum note that has been confusing me – still not sure that all the listed types are stored in 1.1 format since the statement about TXMP was incorrect (unless this was written and true for OS 1.0?)) |
||
| (4 intermediate revisions by the same user not shown) | |||
| Line 3: | Line 3: | ||
{{Hatnote|".dat" redirects here; for other files ending in ".dat", see [[Oni (folder)]].<br> | {{Hatnote|".dat" redirects here; for other files ending in ".dat", see [[Oni (folder)]].<br> | ||
:You should read [[Engine terminology]] and [[Game data terminology]] before this page.<br> | :You should read [[Engine terminology]] and [[Game data terminology]] before this page.<br> | ||
:[[ | :The [[raw|Raw and separate file formats]] page should be read after this one.}} | ||
Files in GameDataFolder/ named "level[0-19]_Final.dat", together with ".raw" and sometimes ".sep" counterparts, contain the game data for Oni. These are called "instance files" internally, but a more common-sense name for them is level data files. The format described below was also used for the tool files which supplied the GUI for the in-game editor, however the retail Oni game application refuses to load tool files; for the story behind the tool files, see [[level0_Tools]]. | Files in GameDataFolder/ named "level[0-19]_Final.dat", together with ".raw" and sometimes ".sep" counterparts, contain the game data for Oni. These are called "instance files" internally, but a more common-sense name for them is level data files. The format described below was also used for the tool files which supplied the GUI for the in-game editor, however the retail Oni game application refuses to load tool files; for the story behind the tool files, see [[level0_Tools]]. | ||
| Line 31: | Line 31: | ||
*0x0003BA70A8DBAE11 - templates compatible with PlayStation 2 engine | *0x0003BA70A8DBAE11 - templates compatible with PlayStation 2 engine | ||
<!--*0x0000000000000000 (blank) - for use with [[OniX]] engine, which instead handles data versioning using the 0x3C field below--> | <!--*0x0000000000000000 (blank) - for use with [[OniX]] engine, which instead handles data versioning using the 0x3C field below--> | ||
OniSplit's .oni files use PC 1.0 checksum by default and 1.1 | OniSplit's .oni files use PC 1.0 checksum by default and 1.1 checksum when storing data that comes from the 1.1 engine and which perceptibly differs in format for that engine (AGQG, IGSt, M3GM, SNDD, TRAM, TSFT, TSGA, but not TXMP) }} | ||
{{OBDtr| 0x08 | uint32 | | 31 33 52 56 | '13RV' | .dat version (meant to be read as "VR31")<br>OniSplit's .oni files use '23RV' ("VR32") instead<!--<br>OniX's [[Oni (folder)|GDFX]] uses '33RV' ("VR33") to signify that the new data versioning system is in use--> }} | {{OBDtr| 0x08 | uint32 | | 31 33 52 56 | '13RV' | .dat version (meant to be read as "VR31")<br>OniSplit's .oni files use '23RV' ("VR32") instead<!--<br>OniX's [[Oni (folder)|GDFX]] uses '33RV' ("VR33") to signify that the new data versioning system is in use--> }} | ||
{{OBDtr| 0x0C | uint16 | | 40 00 | 64 | size of this header }} | {{OBDtr| 0x0C | uint16 | | 40 00 | 64 | size of this header }} | ||
| Line 38: | Line 38: | ||
{{OBDtr| 0x12 | uint16 | | 08 00 | 8 | size of name descriptor }} | {{OBDtr| 0x12 | uint16 | | 08 00 | 8 | size of name descriptor }} | ||
{{OBDtr| 0x14 | uint32 | | 83 24 00 00 | 9347 | instance descriptor count }} | {{OBDtr| 0x14 | uint32 | | 83 24 00 00 | 9347 | instance descriptor count }} | ||
{{OBDtr| 0x18 | uint32 | | D4 1B 00 00 | 7124 | name descriptor count }} | {{OBDtr| 0x18 | uint32 | | D4 1B 00 00 | 7124 | name descriptor count; 0 for .oni files because there is no name desc. table }} | ||
{{OBDtr| 0x1C | uint32 | | 38 00 00 00 | 56 | template descriptor count }} | {{OBDtr| 0x1C | uint32 | | 38 00 00 00 | 56 | template descriptor count; 0 for .oni files because there is no template desc. table }} | ||
{{OBDtr| 0x20 | uint32 | | A0 BC 03 00 | 0x03BCA0 | data block offset }} | {{OBDtr| 0x20 | uint32 | | A0 BC 03 00 | 0x03BCA0 | data block offset }} | ||
{{OBDtr| 0x24 | uint32 | | A0 35 25 00 | 2438560 | data block size }} | {{OBDtr| 0x24 | uint32 | | A0 35 25 00 | 2438560 | data block size }} | ||
{{OBDtr| 0x28 | uint32 | | 40 F2 28 00 | 0x28F240 | name block offset }} | {{OBDtr| 0x28 | uint32 | | 40 F2 28 00 | 0x28F240 | name block offset; will be lower than data block offset for .oni files }} | ||
{{OBDtr| 0x2C | uint32 | | 04 4F 02 00 | 151300 | name block size }} | {{OBDtr| 0x2C | uint32 | | 04 4F 02 00 | 151300 | name block size }} | ||
{{OBDtr| 0x30 | uint32 | | 99 CF 40 00 | (garbage) | used by OniSplit for raw | {{OBDtr| 0x30 | uint32 | | 99 CF 40 00 | (garbage) | used by OniSplit for raw/sep block offset }} | ||
{{OBDtr| 0x34 | uint32 | | 90 4F 63 00 | (garbage) | used by OniSplit for raw | {{OBDtr| 0x34 | uint32 | | 90 4F 63 00 | (garbage) | used by OniSplit for raw/sep block size }} | ||
{{OBDtr| 0x38 | uint32 | | F4 55 5F 00 | (garbage) | unused<!--used by OniX for data versioning; the three high bytes contains the highest data version (timestamp) found in any instance in this .dat; see instance descriptor table's 0x10 for format--> }} | {{OBDtr| 0x38 | uint32 | | F4 55 5F 00 | (garbage) | unused<!--used by OniX for data versioning; the three high bytes contains the highest data version (timestamp) found in any instance in this .dat; see instance descriptor table's 0x10 for format--> }} | ||
{{OBDtr| 0x3C | uint32 | | 90 4F 63 00 | (garbage) | unused<!--used by OniX for content versioning; the three high bytes contain the highest content version (timestamp) found in any instance in this .dat; see instance descriptor table's 0x10 for format--> }} | {{OBDtr| 0x3C | uint32 | | 90 4F 63 00 | (garbage) | unused<!--used by OniX for content versioning; the three high bytes contain the highest content version (timestamp) found in any instance in this .dat; see instance descriptor table's 0x10 for format--> }} | ||
| Line 58: | Line 58: | ||
The '''descriptor counts''' are the sizes of arrays which are coming up in this file: the instance, name and template descriptors. For instance, the size of the instance descriptor array will be 0x2483, or 9,347 items, in length. | The '''descriptor counts''' are the sizes of arrays which are coming up in this file: the instance, name and template descriptors. For instance, the size of the instance descriptor array will be 0x2483, or 9,347 items, in length. | ||
Next we are told the addresses and sizes of the '''data and name | Next we are told the addresses and sizes of the '''data and name block''' in the instance file. The name block simply follows the data block, as you'll see if you add the data block offset plus the data block size, so the name block offset is technically redundant. The name block offset plus the name block size equals the total size of the file since it's the last segment of the file. (However in a .oni file, the order of these blocks is reversed.) | ||
After the name block's size comes four "int"s of '''garbage'''; this is padding in order to align the start of the next segment of the file on a 32-byte boundary. The first two 32-bit fields in this space are, however, used in .oni files generated by OniSplit<!--, and the | After the name block's size comes four "int"s of '''garbage'''; this is padding in order to align the start of the next segment of the file on a 32-byte boundary. The first two 32-bit fields in this space are, however, used in .oni files generated by OniSplit<!--, and the next two 32-bit fields are used by OniX for a new form of template versioning-->. | ||
That concludes the header of the instance file. Immediately after this header we find the instance descriptors array. | That concludes the header of the instance file. Immediately after this header we find the instance descriptors array. | ||
| Line 247: | Line 247: | ||
==Data block== | ==Data block== | ||
The data block occupies the majority of the file and stores all the instance data (though this data sometimes points to the location of more data in a raw/separate file). We peeked at this | The data block occupies the majority of the file and stores all the instance data (though this data sometimes points to the location of more data in a raw/separate file). We peeked at this block before when we looked at the instance descriptor for SUBTsubtitles. The block's starting point is found at the offset given in the header, in this case 0x03BCA0, saving us the trouble of adding up the size of the four preceding segments of the file and then aligning to the next 32-byte boundary. | ||
The reason we'd need to align to 32 bytes is that the start of each instance's record (the instance ID) is always 32 byte-aligned. Thus, even though the template descriptors ended at 0x03BC9C, there are four empty bytes here so that the data block can begin at 0x03BCA0, which divides evenly by 32. This alignment rule also means that the instance-specific data will always start at an offset like 0x0008, 0x0028, 0x0148, etc. | The reason we'd need to align to 32 bytes is that the start of each instance's record (the instance ID) is always 32 byte-aligned. Thus, even though the template descriptors ended at 0x03BC9C, there are four empty bytes here so that the data block can begin at 0x03BCA0, which divides evenly by 32. This alignment rule also means that the instance-specific data will always start at an offset like 0x0008, 0x0028, 0x0148, etc. | ||
| Line 283: | Line 283: | ||
==Name block== | ==Name block== | ||
This final segment of the file stores all the instance names as C-style ASCII strings (terminated by a zero byte). We peeked at this before when we looked at the instance descriptor for SUBTsubtitles. The start of this | This final segment of the file (third-to-last in a .oni, followed by the data and raw/sep blocks) stores all the instance names as C-style ASCII strings (terminated by a zero byte). We peeked at this before when we looked at the instance descriptor for SUBTsubtitles. The start of this block is 32-byte aligned but after that the strings are simply packed end to end, separated only by their null terminator. As with the data block, the name block's starting point is given in the header, in this case 0x28F240. | ||
{{Table}} | {{Table}} | ||
| Line 290: | Line 290: | ||
|} | |} | ||
These names can be up to 63 characters long, counting the tag. The instance file concludes with the end of the name block. | These names can be up to 63 characters long, counting the tag. The instance file concludes with the end of the name block, or the raw/sep block in a .oni file. | ||
{{OBD}} | {{OBD}} | ||