OBD:M3GM: Difference between revisions

From OniGalore
Jump to navigation Jump to search
(PS2 implementation of M3GM (not yet fully understood))
m (changed family)
 
(8 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{OBD_File_Header | type=M3GM | prev=M3GA | next=Mtrl | name=Geometry | family=Generic | align=center}}
{{OBD_File_Header | type=M3GM | prev=M3GA | next=Mtrl | name=Geometry | family=General | align=center}}




Line 15: Line 15:
{{OBDtr|0x18|link    |FFFFC8|01 ED 00 00|237  | link to [[OBD:TXCA|00237-.TXCA]] (vertex UVs) }}
{{OBDtr|0x18|link    |FFFFC8|01 ED 00 00|237  | link to [[OBD:TXCA|00237-.TXCA]] (vertex UVs) }}
{{OBDtr|0x1C|link    |FFFFC8|01 0C 01 00|268  | link to [[OBD:IDXA_M3GM_1|00268-.IDXA]] (triangle strips) }}
{{OBDtr|0x1C|link    |FFFFC8|01 0C 01 00|268  | link to [[OBD:IDXA_M3GM_1|00268-.IDXA]] (triangle strips) }}
{{OBDtr|0x20|link    |FFFFC8|01 11 01 00|273  | link to [[OBD:IDXA_M3GM_2|00273-.IDXA]] (face groupings) }}
{{OBDtr|0x20|link    |FFFFC8|01 11 01 00|273  | link to [[OBD:IDXA_M3GM_2|00273-.IDXA]] (face grouping by normal) }}
{{OBDtr|0x24|link    |FFFFC8|01 D8 00 00|216  | link to 00216-.[[OBD:TXMP|TXMP]] (texture) }}
{{OBDtr|0x24|link    |FFFFC8|01 D8 00 00|216  | link to 00216-.[[OBD:TXMP|TXMP]] (texture) }}
{{OBDtr|0x28|link    |C8FFC8|00 00 00 00|unused| obsolete GMAN (geometry animation) link; never used in Oni}}
{{OBDtr|0x28|link    |C8FFC8|00 00 00 00|unused| obsolete GMAN (geometry animation) link; never used in Oni}}
Line 39: Line 39:
:Geometry animation worked in a way similar to texture animation, by picking M3GMs from an array (GMAN) and displaying them at a specified frame rate, optionally with randomization.
:Geometry animation worked in a way similar to texture animation, by picking M3GMs from an array (GMAN) and displaying them at a specified frame rate, optionally with randomization.


----
;PS2 implementation
;PS2 implementation
:PS2 M3GMs have the same data size as PC and Mac ones (0x24 bytes of actual data, 0x2C if counting the instance and level IDs), but the layout is different. Normals aren't stored at all, hence both VCRA and the second IDXA are simply missing. Instead the M3GM explicitly stores the number of vertices, as well as (apparently) the number of unique XYZ coordinates (a.k.a. control points). There is also a mysterious .sep part (not yet documented).
:PS2 M3GMs have the same data size as PC and Mac ones (0x24 bytes of actual data, 0x2C if counting the instance and level IDs), but the layout is different.
:An an example, consider M3GMaxes of level0_Final (even though it is somewhat broken).
:*Face normals aren't stored at all, and vertex normals aren't stored as VCRA either, therefore both VCRA and the second IDXA are simply missing, which frees up 12 bytes worth of space for data fields unique to the PS2 format.
:*8 of the 12 freed-up bytes are used by explicitly storing the number of vertices (i.e., the array size common to PNTA and TXCA) and the number of faces (same count as the 2nd VCRA on PC and Mac), as two int32's.
:*The other 4 bytes are used to reference the compressed storage of vertex normals in the.raw file. This storage is very compact/lossy, using only 1 byte per normal (instead of 12 bytes per normal in the case of VCRA storage).
:An an example, consider M3GMaxes of level0_Final.
{{Table}}
{{Table}}
{{OBDth}}
{{OBDth}}
Line 47: Line 51:
{{OBDtr|0x04|lev_id  |FFFF00|01 00 00 00|0    | level 0}}
{{OBDtr|0x04|lev_id  |FFFF00|01 00 00 00|0    | level 0}}
{{OBDtr|0x08|int32  |FFC8C8|00 00 00 00|0    | runtime geometry flags (supposedly the same as for PC and Mac) }}
{{OBDtr|0x08|int32  |FFC8C8|00 00 00 00|0    | runtime geometry flags (supposedly the same as for PC and Mac) }}
{{OBDtr|0x0C|int32  |FFC8C8|60 00 00 00|96    | number of vertices with unique XYZ coordinates? }}
{{OBDtr|0x0C|int32  |FFC8C8|60 00 00 00|96    | number of faces (same as in second VCRA on PC and Mac) }}
{{OBDtr|0x10|int32  |FFFFC8|01 64 00 00|100  | link to [[OBD:PNTA|00100-.PNTA]] (vertex XYZs) }}
{{OBDtr|0x10|int32  |FFFFC8|01 64 00 00|100  | link to [[OBD:PNTA|00100-.PNTA]] (vertex XYZs) }}
{{OBDtr|0x14|link    |FFFFC8|01 65 00 00|101  | link to [[OBD:TXCA|00101-.TXCA]] (vertex UVs) }}
{{OBDtr|0x14|link    |FFFFC8|01 65 00 00|101  | link to [[OBD:TXCA|00101-.TXCA]] (vertex UVs) }}
{{OBDtr|0x18|link    |FFFFC8|01 66 00 00|240   | link to [[OBD:IDXA_M3GM_1|00268-.IDXA]] (triangle strips) }}
{{OBDtr|0x18|link    |FFFFC8|01 66 00 00|102   | link to [[OBD:IDXA_M3GM_1|00102-.IDXA]] (triangle strips) }}
{{OBDtr|0x1C|int32  |FFC8C8|6C 00 00 00|108  | number of vertices (same as in PNTA and TXCA); also size of .raw part? }}
{{OBDtr|0x1C|int32  |FFC8C8|6C 00 00 00|108  | number of vertices (same as in PNTA and TXCA); also size of .raw part in bytes }}
{{OBDtr|0x20|offset  |FFFFC8|A0 8E 01 00| 0x00018EA0 | apparently an offset into the .raw file, purpose unknown (108 first bytes shown in bold)
{{OBDtr|0x20|offset  |FFFFC8|A0 8E 01 00| 0x00018EA0 | offset into the .raw file where the the 108 compressed vertex normals are stored:
  '''2D 2D 2D 2D 2D 2D 2D 2D 2D 2D 2D 2D F8 F8 C2 55'''
  '''2D 2D 2D 2D 2D 2D 2D 2D 2D 2D 2D 2D F8 F8 C2 55'''
  '''40 40 10 D4 D4 09 C2 5F 6B 6B 6B 6B 6B 6B 6B 6B'''
  '''40 40 10 D4 D4 09 C2 5F 6B 6B 6B 6B 6B 6B 6B 6B'''
Line 59: Line 63:
  '''EA EA A7 A7 24 EA EA EA EA 24 02 02 02 A7 A7 A7'''
  '''EA EA A7 A7 24 EA EA EA EA 24 02 02 02 A7 A7 A7'''
  '''A7 EF EA EA EA EA 02 02 02 02 24 24 24 24 F8 2D'''
  '''A7 EF EA EA EA EA 02 02 02 02 24 24 24 24 F8 2D'''
  '''2D 2D 2D A7 A7 A7 A7 EA EA EA EA 16''' 02 02 02 02
  '''2D 2D 2D A7 A7 A7 A7 EA EA EA EA 16'''
Current assumption is that they are color indices, but it is unclear what palette is referenced.  
See below for a overview of the compression/decompression rule, i.e., how the 256 values of a compressed normal are mapped to the unit sphere.
}}
}}
{{OBDtr|0x24|link    |FFFFC8|01 67 00 00|103  | broken texture link (there is no instance 103 (!) and the relevant texture is 00104-_axis.[[OBD:TXMP|TXMP]] }}
{{OBDtr|0x24|link    |FFFFC8|01 67 00 00|103  | texture link (to the empty 00103-_AXIS.[[OBD:TXMP|TXMP]]; the actually relevant 00104-_axis.TXMP is orphaned) }}
{{OBDtr|0x28|link    |C8FFC8|00 00 00 00|unused| supposedly the same GMAN link (geometry animation) as for PC and Mac }}
{{OBDtr|0x28|link    |C8FFC8|00 00 00 00|unused| supposedly the same GMAN link (geometry animation) as for PC and Mac }}
{{OBDtr|0x2C|char[20]|C8FFFF|AD DE      |dead  | unused}}
{{OBDtr|0x2C|char[20]|C8FFFF|AD DE      |dead  | unused}}
|}
|}
Even without a full understanding of the .raw part, the current knowledge is sufficient to extract M3GM data and even to export it to other formats (with missing or autogenerated normals).
;OniBrowser being able to load PS2 M3GMs (a mystery of the modern times)
:Somehow OniBrowser is able to display M3GMs from PS2 instance files (except those with textures), even though all the links (PNТA, TXCA, IDXA) are in the wrong positions. Apparently, in the case of M3GM, OniBrowser reads in all the fields past 0x0C as a bunch of non-typed instance links, then identifies their types and populates the PNTA, VCRA, TXCA and IDXA references with the first encountered instance of the matching type.
;Compressed vertex normals
[[Image:Four-fold tetrahedron.jpg|right|160px]]
:It is somewhat uncommon to pack a normal vector into a single byte, but it turns out that there is a relatively straightforward way to subdivide the unit sphere into 256 sectors. One starts by splitting the sphere into four large quadrants (based on tetrahedral symmetry), and then each of the big "triangles" is subdivided into 4, then 16, then 64 smaller triangles, like the picture to the right. There is some distortion near the four poles, but other than that it's a valid way to pack normals if space is an issue (if packing to 2 bytes or more, then one would typically work in plain spherical coordinates instead, i.e. azimuth and elevation).
:There isn't much of a point in giving a detailed description of the mapping here, but the basic idea would be to use the lowest 64 values for the first quadrant, the next 64 values for the second quadrant, etc. In actuality the PS2 implementation uses 63 for the first quadrant, 62 for the second, 63 for the third, and 68 for the last - for whatever reason. Within a quadrant the distribution isn't entirely regular, either. To be documented later if at all.
;"Bad door lighting"
:When looking at the .raw part of the M3GM for some doors (level geometry), a striking observation is that there are only three distinct normal values (0xDE, 0x6A and 0x82), rather than the 6 normals that you'd expect for an axis-aligned box. Apparently half of those are bad, inward-pointing normals that no one ever bothered fixing. The regular VCRA normals for the same doors (on PC and Mac) are messed up in exactly the same way. Possibly this is related to the BSL variable '''door_pop_lighting''' ("uses bad door lighting").




{{OBD_File_Footer | type=M3GM | prev=M3GA | next=Mtrl | name=Geometry | family=Generic | align=center}}
{{OBD_File_Footer | type=M3GM | prev=M3GA | next=Mtrl | name=Geometry | family=General | align=center}}


{{OBD}}
{{OBD}}

Latest revision as of 01:58, 8 December 2023

ONI BINARY DATA
M3GA << Other file types >> Mtrl
M3GM : Geometry
switch to XML:M3GM page
Overview @ Oni Stuff
OBD.png


M3gm all.gif


Offset Type Raw Hex Value Description
0x00 res_id 01 D7 00 00 215 00215-door_1_0.M3GM
0x04 lev_id 01 00 00 06 3 level 3
0x08 int32 00 00 00 00 0 runtime only
0x0C link 01 DA 00 00 218 link to 00218-.PNTA (vertex XYZs)
0x10 link 01 F7 00 00 247 link to 00247-.VCRA (vertex normals)
0x14 link 01 F0 00 00 240 link to 00240-.VCRA (face normals)
0x18 link 01 ED 00 00 237 link to 00237-.TXCA (vertex UVs)
0x1C link 01 0C 01 00 268 link to 00268-.IDXA (triangle strips)
0x20 link 01 11 01 00 273 link to 00273-.IDXA (face grouping by normal)
0x24 link 01 D8 00 00 216 link to 00216-.TXMP (texture)
0x28 link 00 00 00 00 unused obsolete GMAN (geometry animation) link; never used in Oni
0x2C char[20] AD DE dead unused


Vertices
XYZ and UV coordinates are stored in parallel (same number of entries in PNTA and TXCA).
Vertex normals
The first VCRA stores the normals for every vertex (same entries as in PNTA and TXCA).
Vertex normals are used by Gouraud shading (directional lighting).
Face normals
The second VCRA stores the normals for every face (groups defined by the second IDXA).
Face normals are used for backface culling.
Triangle strips
The first IDXA lists the triangles as strips. The IDs are the ones in PNTA and IDXA.
The start of a new strip is signaled by a high bit in the ID of its first vertex.
Strips are more optimal for rendering, they are generated when authoring an M3GM.
Face groupings
The second IDXA groups the triangles into faces (oriented by the second VCRA).
Geometry animation
Geometry animation worked in a way similar to texture animation, by picking M3GMs from an array (GMAN) and displaying them at a specified frame rate, optionally with randomization.

PS2 implementation
PS2 M3GMs have the same data size as PC and Mac ones (0x24 bytes of actual data, 0x2C if counting the instance and level IDs), but the layout is different.
  • Face normals aren't stored at all, and vertex normals aren't stored as VCRA either, therefore both VCRA and the second IDXA are simply missing, which frees up 12 bytes worth of space for data fields unique to the PS2 format.
  • 8 of the 12 freed-up bytes are used by explicitly storing the number of vertices (i.e., the array size common to PNTA and TXCA) and the number of faces (same count as the 2nd VCRA on PC and Mac), as two int32's.
  • The other 4 bytes are used to reference the compressed storage of vertex normals in the.raw file. This storage is very compact/lossy, using only 1 byte per normal (instead of 12 bytes per normal in the case of VCRA storage).
An an example, consider M3GMaxes of level0_Final.
Offset Type Raw Hex Value Description
0x00 res_id 01 63 00 00 99 00099-axes.M3GM
0x04 lev_id 01 00 00 00 0 level 0
0x08 int32 00 00 00 00 0 runtime geometry flags (supposedly the same as for PC and Mac)
0x0C int32 60 00 00 00 96 number of faces (same as in second VCRA on PC and Mac)
0x10 int32 01 64 00 00 100 link to 00100-.PNTA (vertex XYZs)
0x14 link 01 65 00 00 101 link to 00101-.TXCA (vertex UVs)
0x18 link 01 66 00 00 102 link to 00102-.IDXA (triangle strips)
0x1C int32 6C 00 00 00 108 number of vertices (same as in PNTA and TXCA); also size of .raw part in bytes
0x20 offset A0 8E 01 00 0x00018EA0 offset into the .raw file where the the 108 compressed vertex normals are stored:
2D 2D 2D 2D 2D 2D 2D 2D 2D 2D 2D 2D F8 F8 C2 55
40 40 10 D4 D4 09 C2 5F 6B 6B 6B 6B 6B 6B 6B 6B
6B 82 82 D9 16 E3 E3 00 EB EB 6D 6D 6D 6D 6D 6D
6D 6D 6D 6D AB AB EB 80 52 FC 95 30 30 EA EA EA
EA EA A7 A7 24 EA EA EA EA 24 02 02 02 A7 A7 A7
A7 EF EA EA EA EA 02 02 02 02 24 24 24 24 F8 2D
2D 2D 2D A7 A7 A7 A7 EA EA EA EA 16

See below for a overview of the compression/decompression rule, i.e., how the 256 values of a compressed normal are mapped to the unit sphere.

0x24 link 01 67 00 00 103 texture link (to the empty 00103-_AXIS.TXMP; the actually relevant 00104-_axis.TXMP is orphaned)
0x28 link 00 00 00 00 unused supposedly the same GMAN link (geometry animation) as for PC and Mac
0x2C char[20] AD DE dead unused
OniBrowser being able to load PS2 M3GMs (a mystery of the modern times)
Somehow OniBrowser is able to display M3GMs from PS2 instance files (except those with textures), even though all the links (PNТA, TXCA, IDXA) are in the wrong positions. Apparently, in the case of M3GM, OniBrowser reads in all the fields past 0x0C as a bunch of non-typed instance links, then identifies their types and populates the PNTA, VCRA, TXCA and IDXA references with the first encountered instance of the matching type.
Compressed vertex normals
Four-fold tetrahedron.jpg
It is somewhat uncommon to pack a normal vector into a single byte, but it turns out that there is a relatively straightforward way to subdivide the unit sphere into 256 sectors. One starts by splitting the sphere into four large quadrants (based on tetrahedral symmetry), and then each of the big "triangles" is subdivided into 4, then 16, then 64 smaller triangles, like the picture to the right. There is some distortion near the four poles, but other than that it's a valid way to pack normals if space is an issue (if packing to 2 bytes or more, then one would typically work in plain spherical coordinates instead, i.e. azimuth and elevation).
There isn't much of a point in giving a detailed description of the mapping here, but the basic idea would be to use the lowest 64 values for the first quadrant, the next 64 values for the second quadrant, etc. In actuality the PS2 implementation uses 63 for the first quadrant, 62 for the second, 63 for the third, and 68 for the last - for whatever reason. Within a quadrant the distribution isn't entirely regular, either. To be documented later if at all.
"Bad door lighting"
When looking at the .raw part of the M3GM for some doors (level geometry), a striking observation is that there are only three distinct normal values (0xDE, 0x6A and 0x82), rather than the 6 normals that you'd expect for an axis-aligned box. Apparently half of those are bad, inward-pointing normals that no one ever bothered fixing. The regular VCRA normals for the same doors (on PC and Mac) are messed up in exactly the same way. Possibly this is related to the BSL variable door_pop_lighting ("uses bad door lighting").


ONI BINARY DATA
M3GA << Other file types >> Mtrl
M3GM : Geometry
General file