OBD:M3GM: Difference between revisions

OBD:M3GM (view source)

Revision as of 21:26, 20 November 2022

427 bytes added , 20 November 2022

objective complete... ish

Geyser

Administrators

5,389

edits

@@ Line 15: / Line 15: @@
 {{OBDtr|0x18|link    |FFFFC8|01 ED 00 00|237   | link to [[OBD:TXCA|00237-.TXCA]] (vertex UVs) }}
 {{OBDtr|0x1C|link    |FFFFC8|01 0C 01 00|268   | link to [[OBD:IDXA_M3GM_1|00268-.IDXA]] (triangle strips) }}
-{{OBDtr|0x20|link    |FFFFC8|01 11 01 00|273   | link to [[OBD:IDXA_M3GM_2|00273-.IDXA]] (face normal assignment) }}
+{{OBDtr|0x20|link    |FFFFC8|01 11 01 00|273   | link to [[OBD:IDXA_M3GM_2|00273-.IDXA]] (face grouping by normal) }}
 {{OBDtr|0x24|link    |FFFFC8|01 D8 00 00|216   | link to 00216-.[[OBD:TXMP|TXMP]] (texture) }}
 {{OBDtr|0x28|link    |C8FFC8|00 00 00 00|unused| obsolete GMAN (geometry animation) link; never used in Oni}}
@@ Line 41: / Line 41: @@
 ----
 ;PS2 implementation
-:PS2 M3GMs have the same data size as PC and Mac ones (0x24 bytes of actual data, 0x2C if counting the instance and level IDs), but the layout is different. Normals aren't stored at all, hence both VCRA and the second IDXA are simply missing. Instead the M3GM explicitly stores the number of vertices (i.e., the array size common to PNTA and TXCA), as well as another vertex count (not fully understood). There is also a mysterious .raw part (also not documented yet).
+:PS2 M3GMs have the same data size as PC and Mac ones (0x24 bytes of actual data, 0x2C if counting the instance and level IDs), but the layout is different.
+:*Face normals aren't stored at all, and vertex normals aren't stored as VCRA either, therefore both VCRA and the second IDXA are simply missing, which frees up 12 bytes worth of space for data fields unique to the PS2 format.
+:*8 of the 12 freed-up bytes are used by explicitly storing the number of vertices (i.e., the array size common to PNTA and TXCA) and the number of faces (same count as the 2nd VCRA on PC and Mac), as two int32's.
+:*The other 4 bytes are used to reference the compressed storage of vertex normals in the.raw file. This storage is very compact/lossy, using only 1 byte per normal (instead of 12 bytes per normal in the case of VCRA storage).
 :An an example, consider M3GMaxes of level0_Final.
 {{Table}}
@@ Line 48: / Line 51: @@
 {{OBDtr|0x04|lev_id  |FFFF00|01 00 00 00|0     | level 0}}
 {{OBDtr|0x08|int32   |FFC8C8|00 00 00 00|0     | runtime geometry flags (supposedly the same as for PC and Mac) }}
-{{OBDtr|0x0C|int32   |FFC8C8|60 00 00 00|96    | number of actual vertices? (see discussion below) }}
+{{OBDtr|0x0C|int32   |FFC8C8|60 00 00 00|96    | number of faces (same as in second VCRA on PC and Mac) }}
 {{OBDtr|0x10|int32   |FFFFC8|01 64 00 00|100   | link to [[OBD:PNTA|00100-.PNTA]] (vertex XYZs) }}
 {{OBDtr|0x14|link    |FFFFC8|01 65 00 00|101   | link to [[OBD:TXCA|00101-.TXCA]] (vertex UVs) }}
 {{OBDtr|0x18|link    |FFFFC8|01 66 00 00|102   | link to [[OBD:IDXA_M3GM_1|00102-.IDXA]] (triangle strips) }}
 {{OBDtr|0x1C|int32   |FFC8C8|6C 00 00 00|108   | number of vertices (same as in PNTA and TXCA); also size of .raw part in bytes }}
-{{OBDtr|0x20|offset  |FFFFC8|A0 8E 01 00| 0x00018EA0 | offset into the .raw file, purpose unknown (first 108 bytes shown in bold)
+{{OBDtr|0x20|offset  |FFFFC8|A0 8E 01 00| 0x00018EA0 | offset into the .raw file where the the 108 compressed vertex normals are stored:
   '''2D 2D 2D 2D 2D 2D 2D 2D 2D 2D 2D 2D F8 F8 C2 55'''
   '''40 40 10 D4 D4 09 C2 5F 6B 6B 6B 6B 6B 6B 6B 6B'''
@@ Line 60: / Line 63: @@
   '''EA EA A7 A7 24 EA EA EA EA 24 02 02 02 A7 A7 A7'''
   '''A7 EF EA EA EA EA 02 02 02 02 24 24 24 24 F8 2D'''
-  '''2D 2D 2D A7 A7 A7 A7 EA EA EA EA 16''' 02 02 02 02
+  '''2D 2D 2D A7 A7 A7 A7 EA EA EA EA 16'''
-After looking at PS2's implementation of TXMP, it is a fair assumption that they are color indices, but then it is unclear what palette is referenced (grayscale?)
+See below for a overview of the compression/decompression rule, i.e., how the 256 values of a compressed normal are mapped to the unit sphere.
-:It could also be that they're packed normal vectors, although compressing one normal direction into a single byte is rather lossy and coarse.
 }}
 {{OBDtr|0x24|link    |FFFFC8|01 67 00 00|103   | texture link (to the empty 00103-_AXIS.[[OBD:TXMP|TXMP]]; the actually relevant 00104-_axis.TXMP is orphaned) }}
@@ Line 68: / Line 70: @@
 {{OBDtr|0x2C|char[20]|C8FFFF|AD DE      |dead  | unused}}
 |}
-Even without a full understanding of the .raw part, the handling of normals and the vertex count at 0x0C, the current knowledge is sufficient to extract M3GM data from PS2 resources and even to export it to other formats (with missing or autogenerated normals).
 ;OniBrowser being able to load PS2 M3GMs (a mystery of the modern times)
-:Somehow OniBrowser is able to display M3GMs from PS2 instance files (except those with textures), even though all the links (PTNA, TXCA, IDXA) are in the wrong positions. Apparently, in the case of M3GM, OniBrowser reads in all the fields past 0x0C as a bunch of non-typed instance links, then identifies their types and populates the PNTA, VCRA, TXCA and IDXA references as soon (or as late) as a matching type is encountered.
+:Somehow OniBrowser is able to display M3GMs from PS2 instance files (except those with textures), even though all the links (PTNA, TXCA, IDXA) are in the wrong positions. Apparently, in the case of M3GM, OniBrowser reads in all the fields past 0x0C as a bunch of non-typed instance links, then identifies their types and populates the PNTA, VCRA, TXCA and IDXA references with the first encountered instance of the matching type.
-;Number of actual vertices (0x0C)
+;Compressed vertex normals
-:For low-poly meshes this vertex count corresponds to the number or individual XYZ coordinate meshes, i.e., the number of "actual" vertices or "control points", regardless of texture coordinates on adjacent faces. Thus this number is systematically 8 for cube-like meshes, 12, for a two-segment square tube, etc. However, for higher-poly meshes (e.g., character body parts), the vertex count at 0x0C actually exceeds the PNTA/TXCA count, possibly indicating that vertices with split normals (at sharp edges) are counted several times. This number isn't directly tied to other resources, so its meaning can only be deduced from a more careful/massive analysis of meshes.
+:It is somewhat uncommon to pack a normal vector into a single byte, but it turns out that there is a relatively straightforward way to subdivide the unit sphere into 256 sectors. One starts by splitting the sphere into four large quadrants (based on tetrahedral symmetry), and then each of the big "triangles" is subdivided into 4, then 16, then 64 smaller triangles, like [https://www.researchgate.net/publication/338662028/figure/fig8/AS:848495074349060@1579308398315/The-4-fold-tetrahedron-version-Discrete-Global-Grid-Systems-DGGSs-based-on-SACs-level_Q320.jpg this]. There is some distorsion near the poles
-;Packed normals?
+:There isn't much of a point in giving a detailed description of the mapping here, but the basic idea would be to use the lowest 64 values for the first quadrant, the next 64 values for the second quadrant, etc. In actuality the PS2 implementation uses 63 for the first quadrant, 62 for the second, 63 for the third, and 68 for the last - for whatever reason. Within a quadrant the distribution isn't entirely regular, either. To be documented later if at all.
-:Regarding the bytes in the .raw file and how they could be packed normals (e.g., in spherical coordinates) - these days normals are packed into no less that 2 bytes, but Oni's lighting is lackluster overall, so maybe drastically compressed normals were deemed sufficient. The analysis of box-like door meshes M3GMsphere (and maybe some pipes from a level's "furniture") should be able to clarify the format completely.
+:When looking at the .raw part of the M3GM for some doors (level geometry), a striking observation is that there are only three distinct normal values (0xDE, 0x6A and 0x82), rather than the 6 normals that you'd expect for an axis-aligned box. Apparently half of those are bad, inward-pointing normals that no one ever bothered fixing. The regular VCRA normals for the same doors (on PC and Mac) are messed up in exactly the same way. Possibly this is related to the BSL variable '''door_pop_lighting''' ("uses bad door lighting").