19,903
edits
OBD:Instance file format: Difference between revisions
more clarity on the use of name descriptors, including that binary search is not actually taking place
(better wording and example for instance and file ID) |
(more clarity on the use of name descriptors, including that binary search is not actually taking place) |
||
| Line 26: | Line 26: | ||
{{OBD_Table_Header}} | {{OBD_Table_Header}} | ||
{{OBDtr| 0x00 | uint64 | | 1F 27 DC 33 DF BC 03 00 | 0x0003BCDF33DC271F | Total template checksum (main indicator of engine compatibility): | {{OBDtr| 0x00 | uint64 | | 1F 27 DC 33 DF BC 03 00 | 0x0003BCDF33DC271F | Total template checksum (main indicator of engine compatibility): | ||
*0x0003BCDF33DC271F (v1.0) - templates compatible with Windows retail engine | *0x0003BCDF33DC271F (PC v1.0) - templates compatible with Windows retail engine | ||
*0x0003BCDF23C13061 (v1.1) - templates compatible with Windows demo and Mac engines | *0x0003BCDF23C13061 (PC v1.1) - templates compatible with Windows demo and Mac engines | ||
*0x0003BA70A8DBAE11 (PS2) - templates compatible with PlayStation 2 engine | *0x0003BA70A8DBAE11 (PS2) - templates compatible with PlayStation 2 engine | ||
<!--*0x0000000000000000 (blank) - for use with [[OniX]] engine, which instead handle data versioning using the 0x3C field below--> | <!--*0x0000000000000000 (blank) - for use with [[OniX]] engine, which instead handle data versioning using the 0x3C field below--> | ||
| Line 213: | Line 213: | ||
The name descriptor array starts immediately after the instance descriptors array. To find the end of the instance descriptors, we can simply take the size of an instance descriptor, 20 bytes, and multiply it by the number of instance descriptors in the file header. In this case, that means 20 * 9347 = 186940, or 0x02DA3C. Adding that to 0x40 (the start of the instance descriptors) takes us to address 0x02DA7C. Voila, the start of the name descriptors. | The name descriptor array starts immediately after the instance descriptors array. To find the end of the instance descriptors, we can simply take the size of an instance descriptor, 20 bytes, and multiply it by the number of instance descriptors in the file header. In this case, that means 20 * 9347 = 186940, or 0x02DA3C. Adding that to 0x40 (the start of the instance descriptors) takes us to address 0x02DA7C. Voila, the start of the name descriptors. | ||
The name descriptor array stores the numbers of all named instances in alphabetical order. This | The name descriptor array stores the numbers of all named instances in the alphabetical order by said names, which are found in the name table but also pointed to by these entries. This array is used by the engine to look up instances by name; it's also used to find instances by template (scanning just the tag at the start of each name). The purpose of this array being alphabetized was to allow the engine to do a binary search to find instances by name more quickly, but the retail engine no longer attempts a binary search and merely iterates over the array from start to end. | ||
{{Table}} | {{Table}} | ||
{{OBD_Table_Header}} | {{OBD_Table_Header}} | ||
{{OBDtr| 0x00 | int32 | | 15 16 00 00 | 5653 | instance | {{OBDtr| 0x00 | int32 | | 15 16 00 00 | 5653 | instance descriptor index }} | ||
{{OBDtr| 0x04 | int32 | | 60 2C 1C 0E | (garbage) | runtime: pointer to instance name }} | {{OBDtr| 0x04 | int32 | | 60 2C 1C 0E | (garbage) | runtime: pointer to instance name }} | ||
|} | |} | ||
The index number here is referring to the instance's position in the instance descriptor array. This number is also used by the data table to identify each instance, thus it is found in two places in the data explicitly and one place implicitly. | |||
Since the addresses of the names in memory cannot be known until the file is loaded into RAM, a space of 32 bits is reserved for each pointer at runtime. | |||
==Template descriptors== | ==Template descriptors== | ||
| Line 250: | Line 254: | ||
{{Table}} | {{Table}} | ||
{{OBD_Table_Header}} | {{OBD_Table_Header}} | ||
{{OBDtr| 0x00 | res_id | | 01 0B 04 00 | 1035 | instance | {{OBDtr| 0x00 | res_id | | 01 0B 04 00 | 1035 | instance descriptor index }} | ||
{{OBDtr| 0x04 | lev_id | | 01 00 00 06 | 3 | level | {{OBDtr| 0x04 | lev_id | | 01 00 00 06 | 3 | level number }} | ||
{{OBDtr| 0x08 | ... | | ... | ... | [[OBD:File types|type-specific data]]... }} | {{OBDtr| 0x08 | ... | | ... | ... | [[OBD:File types|type-specific data]]... }} | ||
|} | |} | ||
| Line 257: | Line 261: | ||
This example is taken from level 3 so that the file ID is more instructive. In the OBD documentation, these fields are called res_id and lev_id as seen above. | This example is taken from level 3 so that the file ID is more instructive. In the OBD documentation, these fields are called res_id and lev_id as seen above. | ||
The '''instance's ID''' is stored as "(instance descriptor index << 8) | 1". Thus the 1,035th entry in the instance descriptor index will be encoded as 0x40B00. The '1' allows the engine to know which IDs have already been converted to pointers (an instance pointer will always be 8-byte aligned, so it will never have the zero bit already set). These pointer flags were retained when the file was written to disk but are meaningless now. At level-load time the flags are cleared and then set again when Oni allocates memory for each instance. The purpose of left-shifting the index number | The '''instance's ID''' is stored as "(instance descriptor index << 8) | 1". Thus the 1,035th entry in the instance descriptor index will be encoded as 0x40B00. The '1' allows the engine to know which IDs have already been converted to pointers (an instance pointer will always be 8-byte aligned, so it will never have the zero bit already set). These pointer flags were retained when the file was written to disk but are meaningless now. At level-load time the flags are cleared and then set again when Oni allocates memory for each instance. The purpose of left-shifting the index number is simply to leave the lowest byte open for the pointer flag. | ||
The '''file ID''' is computed from the number found in the name of the instance file: "(level number << 25) | 1". Thus instances found in level3_Final.dat will have the file ID encoded as 0x6000001. Again, the '1' is used by the engine to know which file IDs have been converted to pointers at runtime, but on disk this is a relic which has no meaning to us. The reason for left-shifting the level number might have been to store it alongside the instance ID and the pointer flag in a single int32, but they are separate numbers now, perhaps so that both IDs can have their own pointer flag. | The '''file ID''' is computed from the number found in the name of the instance file: "(level number << 25) | 1". Thus instances found in level3_Final.dat will have the file ID encoded as 0x6000001. Again, the '1' is used by the engine to know which file IDs have been converted to pointers at runtime, but on disk this is a relic which has no meaning to us. The reason for left-shifting the level number might have originally been to store it alongside the instance ID and the pointer flag in a single int32, but they are separate numbers now, perhaps so that both IDs can have their own pointer flag. | ||
After the header, the size of each instance's data is of a somewhat arbitrary length depending on the template this instance falls under. As mentioned under "Instance descriptors", the data size given by the descriptor includes the 8-byte resource header and the padding at the end of the data to align the next instance on 32 bytes. | After the header, the size of each instance's data is of a somewhat arbitrary length depending on the template this instance falls under. As mentioned under "Instance descriptors", the data size given by the descriptor includes the 8-byte resource header and the padding at the end of the data to align the next instance on 32 bytes. | ||