m (Update to 0.9.90) |
(reporting a pretty important bug, 18 years late) |
||
| (30 intermediate revisions by 4 users not shown) | |||
| Line 1: | Line 1: | ||
Talk page archives: [[/Archive1|#1]] | Talk page archives: [[/Archive1|#1]] | ||
== | ==Textures are darkened when extracted from formats with sub-8-bit channels== | ||
OniSplit | When OniSplit reads the game's textures in any format with <8-bit channel sizes, namely ARGB4444, RGB555 and ARGB1555, and expands the 4- or 5-bit R, G or B channel to 8 bits in response to the {{nbhy}}extract command, it is simply zero-padding the lower bits which results in an overall darkening of the image. The clearest way to see this is with TXMPDAODAN_SHIELD in level0, where all the pixel data in the original RGB555 format says "FF FF". That's meant to be pure white, right? Well when OniSplit extracts this to PNG, etc., it converts the value 11111 found in each channel to 11111000 or 0xF8, resulting in pixels of the color {248, 248, 248} which is a noticeably off-white color. No texture will be able to contain a brighter color than that using OniSplit's math where 11111 << 3 = 11111000. Looking at a histogram of any extracted texture should bear this out, but it's also something the naked eye can see pretty easily if there's bright colors in the image and you have an accurate reference version of the image being extracted. Apparently the community has been missing this ~3% darkening all these years? | ||
Standard practice in this situation is to use "bit replication" where the most significant bits are copied into the space left behind when a value is left-shifted, e.g. a 5-bit channel value 00101 becomes 00101001, not 00101000. This approach properly spreads out the resulting colors across the entire 0-255 range, allowing the 2-byte pixel value FF FF to become pure white while also preserving 00 00 as pure black. There is another scaling algorithm which is even more accurate but slower. | |||
I believe that any new/edited textures passed in through OniSplit will also experience slight darkening, but only if they are converted to one of the above formats with <8-bit channels. Extracted textures that were darkened by being converted to PNG, etc. will regain their brightness when imported back in with {{nbhy}}create because e.g. 11111000 simply goes back to 11111, and this will be displayed by Oni as full-white on a 32-bit display. This may be why no one noticed the issue before now, but probably the key reason is that modders prefer to create every texture in RGB888. But if a user imports an image using the {{nbhy}}format option to create the TXMP.oni in ARGB4444, RGB555 or ARGB1555, it looks like OniSplit will convert it with a downward bias: 0-7 → 0, 8-15 → 1, and finally 248-255 → 31 because it is simply performing a ">> 3" on the number. This is known to darken the image, so the standard practice is to use "rounded scaling" to create a center bias, which I won't detail here. Just wanted to make a note that if someone is fixing the 4-/5-bit → 8-bit conversion math for OniSplit, the math for converting 8-bit → 4-/5-bit should be looked at as well. --[[User:Iritscen|Iritscen]] ([[User talk:Iritscen|talk]]) 23:50, 5 April 2026 (UTC) | |||
==Smaller version steps possible== | |||
AssemblyInfo.cs | |||
//overwrite product version with 5 possible numbers | |||
[assembly: AssemblyInformationalVersion("0.9.99.0.1")] | |||
Utils.cs | |||
#if NETCORE | |||
version = typeof(Utils).GetTypeInfo().Assembly.GetName().Version.ToString(); | |||
#else | |||
//version = typeof(Utils).Assembly.GetName().Version.ToString(); | |||
Assembly assembly = Assembly.GetExecutingAssembly(); | |||
FileVersionInfo fileVersionInfo = FileVersionInfo.GetVersionInfo(assembly.Location); | |||
version = fileVersionInfo.ProductVersion; | |||
#endif | |||
OniSplit -version | |||
0.9.99.0.1 | |||
--[[User:Paradox-01|Paradox-01]] 09:58, 14 November 2023 | |||
==Page improvement== | |||
Shouldn't there be an animation section which includes the options -anim-body, -anim-merge and -blender? --[[User:Paradox-01|paradox-01]] ([[User talk:Paradox-01|talk]]) 00:24, 16 May 2022 (CEST) | |||
:Yes, it's odd that -anim-body and -anim-merge are not documented here, someone should do something about that. <s>-blender arguably does not exist yet (it's in geyser's upcoming 1.0 release and is mentioned on the [[OniSplit/WIP]] page).</s> My bad, -blender was added in 0.9.99.2. --[[User:Iritscen|Iritscen]] ([[User talk:Iritscen|talk]]) 01:05, 16 May 2022 (CEST) | |||
OniSplit | ==If OniSplit refuses to be used== | ||
delete file at AE/Tools/OniSplit.exe.config --[[User:Paradox-01|Paradox-01]] 18:00, 28 June 2020 | |||
==OniSplit version control - getting there...== | |||
The SVN version of OniSplit is currently labeled 0.9.99 (same as the source received from Neo), and at this point it is different from the non-synced 0.9.99.2 that I distributed through EdT. | |||
*"0.9.99" has only two commits (44.1 kHz SNDD export - affecting only electric spark sounds - and other minor SNDD fixes and features); | |||
*0.9.99.2 has a few fixes to 0.9.99 (adjustment to XML parsing rules) and new functionality for Blender import/export of animations; also some new bugs, apparently. | |||
To restore some order, I am going to merge my 0.9.99.2 fixes to SVN, then address the known bugs, and push the version number to 0.9.99.3 - and use SVN-synced development from then onwards. | |||
I am ''not'' going to create a branch for my fixes/improvements, except maybe when it comes to adding something big, like C# FBX support. If you object to this, speak now. --[[User:Geyser|geyser]] ([[User talk:Geyser|talk]]) 00:09, 29 May 2020 (CEST) | |||
: | List of bugfixes and improvements planned for 0.9.99.3 (feel free to add to the list if I forgot something) | ||
*some running/walking anims end up upside-down after reimported into Oni | |||
*-extract:dae -noanim -blender used on a TRBS, make sure the output .dae file has a TRBS prefix | |||
*stairs_markers.dae not imported properly? (and more generally secondary/auxiliary .dae files not imported for levels?) | |||
*TRAM export: support writing of sparse keyframe data (as in Oni), as opposed to writing rotations for ''every'' frame (interpolated) | |||
*maybe allow the export of named TRCM (without material) | |||
*TGA transparency bug? (not-yet-confirmed as a bug; supposedly happens only for SketchUp-made levels; see below) | |||
*various SNDD glitches (inconsistent handling of ADPCM headers) | |||
---- | |||
When will you update the repo? --[[User:Paradox-01|paradox-01]] ([[User talk:Paradox-01|talk]]) 16:40, 13 April 2023 (CEST) | |||
I couldn't import FILM via '''xml master file for level import''' with current Onisplit compiled from svn. A possible fix is to change <code>ReadElementContentAsString</code> to <code>ReadContentAsString</code>. | |||
... | |||
<Films> | |||
<Import>films/BomberKonRun02.xml</Import> | |||
</Films> | |||
... | |||
FilmImporter.cs | |||
... | |||
private void ReadFilms(XmlReader xml, string basePath) | |||
... | |||
//string filePath = Path.Combine(basePath, xml.ReadElementContentAsString()); | |||
string filePath = Path.Combine(basePath, xml.ReadContentAsString()); | |||
:[[User:Paradox-01|paradox-01]] ([[User talk:Paradox-01|talk]]) 12:59, 14 November 2023 (CET) | |||
==TGA Transparency bug== | |||
I extracted HQ_DOUBLED_GLASS as TGA format from vanilla Level8_Final with the OniSplit command -extract:tga. The file was opened in Photoshop and the alpha channel was visible. | |||
I then added that texture to Sketchup and applied it in a test level, it appeared transparent in Sketchup. I used the command -create:level to create a level in Oni. In the game Oni, the texture appeared opaque not transparent. When, the file TXMPHQ_DOUBLED_GLASS.oni was extracted from the test level as TGA, the alpha channel was gone or opaque. | |||
: | |||
Even if the following code was added to the textures.xml file, the texture still appeared opaque in game. | |||
<Texture Name="HQ_DOUBLED_GLASS"> | |||
<GunkFlags>Transparent TwoSided</GunkFlags> | |||
<Format>RGBA</Format> | |||
<Image>images/HQ_DOUBLED_GLASS.tga</Image> | |||
</Texture> | |||
However, if HQ_DOUBLED_GLASS.tga is converted to PNG format with alpha channel, and then the PNG file is used in Sketchup, then the texture will be transparent in game. | |||
: | This occurs using OniSplit version 0.9.96.0 and previous versions, tested to version 0.9.90.0. (EDIT: Transparency loss is observed for all glass textures, not just HQ_DOUBLED_GLASS.) | ||
[[User:EdT|EdT]] 15:03, 14 May 2020 (PDST) | |||
When you described the issue to me in PM, I didn't notice/understand the key part about applying the texture in SketchUp and then using -create:level on SketchUp's output. I thought we were talking about a simple TXMP roundtrip through TGA. Can you confirm that the texture's alpha goes opaque (or disappears) if you use -extract:tga on TXMPHQ_DOUBLED_GLASS.oni, and then immediately reimport it either with '''-create''' (either from XML with extra flags or from TGA)? --[[User:Geyser|geyser]] ([[User talk:Geyser|talk]]) 11:44, 15 May 2020 (CEST) | |||
Regarding the TXMP roundtrip, after extracting TXMPHQ_DOUBLED_GLASS.oni as TGA, I used the command -create:txmp -format:bgra32 -large -genmipmaps, the resulting file had the alpha channel. To confirm, I replaced the file TXMPHQ_DOUBLED_GLASS.oni from the -create:level with the one from -create:txmp in the test level and it was transparent. | |||
[[User:EdT|EdT]] 12:35, 15 May 2020 (PDST) | |||
: | This looks like an issue with the SketchUp pipeline, rather than with OniSplit, since it is only '''-create:level''' that generates a non-transparent texture, and '''-create:txmp''' actually works fine. Can you: a) provide (in PM) the level data that you are using '''-create:level''' on (supposedly a .dae file, some texture files, and some XMLs)? b) provide an example of a level that imports with transparent glass as expected (supposedly a level that didn't come from SketchUp). Thanks. --[[User:Geyser|geyser]] ([[User talk:Geyser|talk]]) 16:46, 16 May 2020 (CEST) | ||
==Sound export bug== | |||
As discussed on Discord there's a sound export bug for SNDDzap*.oni | |||
One might record the sounds in OBS and compare it with the exported ones to generate further hints on what went wrong. --[[User:Paradox-01|paradox-01]] ([[User talk:Paradox-01|talk]]) 13:29, 21 July 2019 (CEST) | |||
#The zap sounds are randomized in an OSBD group, so recording them from the game in some identifiable way would require some extra work (custom OSBD). It's much easier to ask Mac folks for AIFF versions of those sounds. ''And actually, there are [[seven]] zap sounds in the PC demo as well.'' | |||
#OniSplit slaps a 22.05 kHz header on those files, although they're 44.1kHz and have a perfectly good 44.1kHz header in Oni. Not sure why they're getting a 22.05kHz header. Perhaps Neo got confused by the sloppy documentation of the [[OBD:SNDD/wav|WAV header]], which until my edit gave the same hex listing for the three WAV header types. Either that, or he just assumes the sample rate to be 22.05kHz by default and doesn't update it from the actual file. Will check in the code. ''Actually yeah, he just does as if all the sounds were 22.05kHz.'' | |||
#Besides SNDDzap*.oni, there is one other 44.1kHz sound in PC Oni, SNDDap_hit_shld.aif; which suffers from the same export problem, although it's less noticeable. | |||
#Apparently 22.05 kHz stereo sounds are correctly exported as stereo, it's just 44.1 kHz mono files that get a wrong WAV header. | |||
#Will be fixing this in the latest "nightly" OniSplit (haven't yet decided on a version numbering and source control scheme). | |||
#:[[User:Geyser|geyser]] ([[User talk:Geyser|talk]]) 07:15, 25 March 2020 | |||
[[Category:Completed modding tools]][[Category:Bi-platform modding tools]] | |||
Latest revision as of 23:50, 5 April 2026
Talk page archives: #1
Textures are darkened when extracted from formats with sub-8-bit channels
When OniSplit reads the game's textures in any format with <8-bit channel sizes, namely ARGB4444, RGB555 and ARGB1555, and expands the 4- or 5-bit R, G or B channel to 8 bits in response to the ‑extract command, it is simply zero-padding the lower bits which results in an overall darkening of the image. The clearest way to see this is with TXMPDAODAN_SHIELD in level0, where all the pixel data in the original RGB555 format says "FF FF". That's meant to be pure white, right? Well when OniSplit extracts this to PNG, etc., it converts the value 11111 found in each channel to 11111000 or 0xF8, resulting in pixels of the color {248, 248, 248} which is a noticeably off-white color. No texture will be able to contain a brighter color than that using OniSplit's math where 11111 << 3 = 11111000. Looking at a histogram of any extracted texture should bear this out, but it's also something the naked eye can see pretty easily if there's bright colors in the image and you have an accurate reference version of the image being extracted. Apparently the community has been missing this ~3% darkening all these years?
Standard practice in this situation is to use "bit replication" where the most significant bits are copied into the space left behind when a value is left-shifted, e.g. a 5-bit channel value 00101 becomes 00101001, not 00101000. This approach properly spreads out the resulting colors across the entire 0-255 range, allowing the 2-byte pixel value FF FF to become pure white while also preserving 00 00 as pure black. There is another scaling algorithm which is even more accurate but slower.
I believe that any new/edited textures passed in through OniSplit will also experience slight darkening, but only if they are converted to one of the above formats with <8-bit channels. Extracted textures that were darkened by being converted to PNG, etc. will regain their brightness when imported back in with ‑create because e.g. 11111000 simply goes back to 11111, and this will be displayed by Oni as full-white on a 32-bit display. This may be why no one noticed the issue before now, but probably the key reason is that modders prefer to create every texture in RGB888. But if a user imports an image using the ‑format option to create the TXMP.oni in ARGB4444, RGB555 or ARGB1555, it looks like OniSplit will convert it with a downward bias: 0-7 → 0, 8-15 → 1, and finally 248-255 → 31 because it is simply performing a ">> 3" on the number. This is known to darken the image, so the standard practice is to use "rounded scaling" to create a center bias, which I won't detail here. Just wanted to make a note that if someone is fixing the 4-/5-bit → 8-bit conversion math for OniSplit, the math for converting 8-bit → 4-/5-bit should be looked at as well. --Iritscen (talk) 23:50, 5 April 2026 (UTC)
Smaller version steps possible
AssemblyInfo.cs
//overwrite product version with 5 possible numbers
[assembly: AssemblyInformationalVersion("0.9.99.0.1")]
Utils.cs
#if NETCORE
version = typeof(Utils).GetTypeInfo().Assembly.GetName().Version.ToString();
#else
//version = typeof(Utils).Assembly.GetName().Version.ToString();
Assembly assembly = Assembly.GetExecutingAssembly();
FileVersionInfo fileVersionInfo = FileVersionInfo.GetVersionInfo(assembly.Location);
version = fileVersionInfo.ProductVersion;
#endif
OniSplit -version
0.9.99.0.1
--Paradox-01 09:58, 14 November 2023
Page improvement
Shouldn't there be an animation section which includes the options -anim-body, -anim-merge and -blender? --paradox-01 (talk) 00:24, 16 May 2022 (CEST)
- Yes, it's odd that -anim-body and -anim-merge are not documented here, someone should do something about that.
-blender arguably does not exist yet (it's in geyser's upcoming 1.0 release and is mentioned on the OniSplit/WIP page).My bad, -blender was added in 0.9.99.2. --Iritscen (talk) 01:05, 16 May 2022 (CEST)
If OniSplit refuses to be used
delete file at AE/Tools/OniSplit.exe.config --Paradox-01 18:00, 28 June 2020
OniSplit version control - getting there...
The SVN version of OniSplit is currently labeled 0.9.99 (same as the source received from Neo), and at this point it is different from the non-synced 0.9.99.2 that I distributed through EdT.
- "0.9.99" has only two commits (44.1 kHz SNDD export - affecting only electric spark sounds - and other minor SNDD fixes and features);
- 0.9.99.2 has a few fixes to 0.9.99 (adjustment to XML parsing rules) and new functionality for Blender import/export of animations; also some new bugs, apparently.
To restore some order, I am going to merge my 0.9.99.2 fixes to SVN, then address the known bugs, and push the version number to 0.9.99.3 - and use SVN-synced development from then onwards.
I am not going to create a branch for my fixes/improvements, except maybe when it comes to adding something big, like C# FBX support. If you object to this, speak now. --geyser (talk) 00:09, 29 May 2020 (CEST)
List of bugfixes and improvements planned for 0.9.99.3 (feel free to add to the list if I forgot something)
- some running/walking anims end up upside-down after reimported into Oni
- -extract:dae -noanim -blender used on a TRBS, make sure the output .dae file has a TRBS prefix
- stairs_markers.dae not imported properly? (and more generally secondary/auxiliary .dae files not imported for levels?)
- TRAM export: support writing of sparse keyframe data (as in Oni), as opposed to writing rotations for every frame (interpolated)
- maybe allow the export of named TRCM (without material)
- TGA transparency bug? (not-yet-confirmed as a bug; supposedly happens only for SketchUp-made levels; see below)
- various SNDD glitches (inconsistent handling of ADPCM headers)
When will you update the repo? --paradox-01 (talk) 16:40, 13 April 2023 (CEST)
I couldn't import FILM via xml master file for level import with current Onisplit compiled from svn. A possible fix is to change ReadElementContentAsString to ReadContentAsString.
...
<Films>
<Import>films/BomberKonRun02.xml</Import>
</Films>
...
FilmImporter.cs ... private void ReadFilms(XmlReader xml, string basePath) ... //string filePath = Path.Combine(basePath, xml.ReadElementContentAsString()); string filePath = Path.Combine(basePath, xml.ReadContentAsString());
- paradox-01 (talk) 12:59, 14 November 2023 (CET)
TGA Transparency bug
I extracted HQ_DOUBLED_GLASS as TGA format from vanilla Level8_Final with the OniSplit command -extract:tga. The file was opened in Photoshop and the alpha channel was visible. I then added that texture to Sketchup and applied it in a test level, it appeared transparent in Sketchup. I used the command -create:level to create a level in Oni. In the game Oni, the texture appeared opaque not transparent. When, the file TXMPHQ_DOUBLED_GLASS.oni was extracted from the test level as TGA, the alpha channel was gone or opaque.
Even if the following code was added to the textures.xml file, the texture still appeared opaque in game.
<Texture Name="HQ_DOUBLED_GLASS">
<GunkFlags>Transparent TwoSided</GunkFlags>
<Format>RGBA</Format>
<Image>images/HQ_DOUBLED_GLASS.tga</Image>
</Texture>
However, if HQ_DOUBLED_GLASS.tga is converted to PNG format with alpha channel, and then the PNG file is used in Sketchup, then the texture will be transparent in game.
This occurs using OniSplit version 0.9.96.0 and previous versions, tested to version 0.9.90.0. (EDIT: Transparency loss is observed for all glass textures, not just HQ_DOUBLED_GLASS.)
EdT 15:03, 14 May 2020 (PDST)
When you described the issue to me in PM, I didn't notice/understand the key part about applying the texture in SketchUp and then using -create:level on SketchUp's output. I thought we were talking about a simple TXMP roundtrip through TGA. Can you confirm that the texture's alpha goes opaque (or disappears) if you use -extract:tga on TXMPHQ_DOUBLED_GLASS.oni, and then immediately reimport it either with -create (either from XML with extra flags or from TGA)? --geyser (talk) 11:44, 15 May 2020 (CEST)
Regarding the TXMP roundtrip, after extracting TXMPHQ_DOUBLED_GLASS.oni as TGA, I used the command -create:txmp -format:bgra32 -large -genmipmaps, the resulting file had the alpha channel. To confirm, I replaced the file TXMPHQ_DOUBLED_GLASS.oni from the -create:level with the one from -create:txmp in the test level and it was transparent. EdT 12:35, 15 May 2020 (PDST)
This looks like an issue with the SketchUp pipeline, rather than with OniSplit, since it is only -create:level that generates a non-transparent texture, and -create:txmp actually works fine. Can you: a) provide (in PM) the level data that you are using -create:level on (supposedly a .dae file, some texture files, and some XMLs)? b) provide an example of a level that imports with transparent glass as expected (supposedly a level that didn't come from SketchUp). Thanks. --geyser (talk) 16:46, 16 May 2020 (CEST)
Sound export bug
As discussed on Discord there's a sound export bug for SNDDzap*.oni
One might record the sounds in OBS and compare it with the exported ones to generate further hints on what went wrong. --paradox-01 (talk) 13:29, 21 July 2019 (CEST)
- The zap sounds are randomized in an OSBD group, so recording them from the game in some identifiable way would require some extra work (custom OSBD). It's much easier to ask Mac folks for AIFF versions of those sounds. And actually, there are seven zap sounds in the PC demo as well.
- OniSplit slaps a 22.05 kHz header on those files, although they're 44.1kHz and have a perfectly good 44.1kHz header in Oni. Not sure why they're getting a 22.05kHz header. Perhaps Neo got confused by the sloppy documentation of the WAV header, which until my edit gave the same hex listing for the three WAV header types. Either that, or he just assumes the sample rate to be 22.05kHz by default and doesn't update it from the actual file. Will check in the code. Actually yeah, he just does as if all the sounds were 22.05kHz.
- Besides SNDDzap*.oni, there is one other 44.1kHz sound in PC Oni, SNDDap_hit_shld.aif; which suffers from the same export problem, although it's less noticeable.
- Apparently 22.05 kHz stereo sounds are correctly exported as stereo, it's just 44.1 kHz mono files that get a wrong WAV header.
- Will be fixing this in the latest "nightly" OniSplit (haven't yet decided on a version numbering and source control scheme).