Talk:Mod Tool/OniTools addon

From OniGalore
< Talk:Mod Tool
Revision as of 22:50, 28 December 2016 by Paradox-01 (talk | contribs) (level reimports: automatic gunk flag recovery will be also laborious)
Jump to navigation Jump to search

Note to self:

I must merge this with http://wiki.oni2.net/Talk:Mod_Tool
and clean up notes that are out-of-date
--paradox-01 (talk) 21:27, 16 July 2014 (CEST)


Dev notes for the remake

Currently working on: OBOA-to-Physics.xml


Combined checklist of addon and GUI

estimated time
XSI addon features (1y)
small GUI features and editors (1y)
BSL and XML editor (1y)
fine-tuning, bug-fixes, refactoring (1y)
demo-level with prone-mode (1y)
wth ...


convert level dat to project

  1. create project name by AKEV name [x]
  2. create master file [x]
  3. copy BSL folder []
  4. populate master file with object collection [x]
  5. populate master file with films [x]
  6. populate master file with cameras [] (not all OBAN are cams, have to examine BSL)
  7. get sky name [x]
  8. connect master with shared classes [x]
  9. transfer ONLV door locklights to particle collection [x]
  10. transfer animated objects to physics file []
  11. populate master file with "script" geometry [x] (those are named LevelName_script_scriptId.dae, static parts of animated objects)
  12. transfer spawnable geometry to master file []
count: 12


create level project from scratch

  1. create project folder [x]
  2. create bsl folder [x]
  3. create ONLD (for level number and savegame name) [x]
  4. create master file [x]
  5. set default sky for fast tests [x]
  6. set default player for fast tests [x]
  7. set default ground and PF for fast tests []
  8. set default texture quality [x]
  9. set path to classes folder in master file [x]
count: 9


level finalization

  1. patch final ONLV physics with OFGA particles [] (Physics.xml is equivalent to OBOA except it imports *.dae instead of OFGA*.oni)
  2. check dependencies []
  3. check for deco (TXMB) []
  4. check for logic (BSL main func) []
  5. check for package information []
  6. convert project to package []
count: 6


interaction

  1. xsi user cam movement is monitored by GUI []
  2. GUI plays sounds when cam is within range of sound spheres in xsi []
count: 2
difficulty: medium


file management and editors

  1. detect Oni retail* version []
  2. allow classes collection for retail version (extended version of Neo's collection) []
  3. install file watcher for classes folder []
  4. creating classes in xsi auto-updates html object library []
  5. xsi vertex shading editor (with easy controls for light sources) []
  6. decal particle editor []
  7. xml editor []
  8. populate xml editor with auto-updating classes cache []
  9. BSL editor []
  10. populate BSL editor with auto-updating classes cache []
  11. load script functions of TV, CONS, DOOR, TRIG, TURR on demand []
count: 11
difficulty: medium - high
* GUI searches for a few binary signatures that are only present in retail version. The signatures are hashed so that decompiling the GUI won't reveal the actual search strings. The downloaded file is then automatically decrypted.


convenient features

  1. make GUI an AE package []
  2. setup XSI for Oni modding (non-transparent CHAR tex, frame rate) []
  3. get package numbers from Oni Mod Depot []
  4. integrate TRAM assistant []
  5. SNDD assistant []
  6. pages assistant []
  7. fly-in image assistant (for BSL) []
  8. file association xml []
  9. file association oni []
  10. file association bsl []
  11. file association dat []
  12. gunk flag recovery for special textures (ONLV's ONOA and AKEV's AGQG and AKEV's texture list must be read for this) [] (laborious)
count: 12
difficulty: high


addon

  1. read, write, edit CHAR [][][]
  2. read, write, edit CONS [][][]
  3. read, write, edit DOOR [][][]
  4. read, write, edit ENV markers [][][]
  5. read, write, edit FLAG [][][]
  6. read, write, edit FURN [][] (original FURN is part of AKEV core geometry)
  7. read, write, edit NEUT [][][]
  8. read, write, edit PART [][][]
  9. read, write, edit PATR [][][]
  10. read, write, edit PWRU [][][]
  11. read, write, edit SNDG [][][]
  12. read, write, edit TRIG [][][]
  13. read, write, edit TRGV [][][]
  14. read, write, edit TURR [][][]
  15. read, write, edit WEAP [][][]
  16. add new corpses []
  17. material-depending polygon deletion []
count: 46


features that are unlikely to happen

  1. edit old corpses
  2. automatic sound creation
  3. full-featured particle editor
  4. CHAR rendering in Blender
  5. file context menu xml (multiple options) []
  6. folder context menu (built dat) []



New code snippets

Latest MT addon attempt. Reducing complexity by dropping support for adding AE resources. Vanilla material only.

Download link to the "shared" collection should only provided if Oni retail installation is found on client machine.

Dragging and dropping items from Oni Object Library into MT works with Firefox but not Google Chrome. Have to look into this. 

<a href="characters/ONCC/ONCCbarabus.xml">
	<div class="element-item Syndicate Barabas Daodan Boss Male" id="img" style="background-image: url(characters/ONCC/ONCCbarabus.png);">
		<p class="name">barabus</p>
	</div>
 </a>
Oni Object Library, character page

The relative path of the xml is transformed into an absolute path when dropped into MT.

If the link doesn't begin with file:/// MT will add it. I removed it from html file so xml can be clicked and viewed.

Scratching together... 

x = done
- = not needed
- characters consoles doors env markers furniture ghost quads powerups triggers turrets weapons
shared (collection) x x x x x - x x x x
OniToDae + Html html - html x x html
preview imgs x x - x x x
DnD recognition x - x x x
layer support x x x x x
PPG interface x x - x -
save in project
PrjToXml

more ... later

flags, patrol path, tvs, crsa, sound, script objects (spawnable geometry)

Patrol paths are the trickiest. There's no easy way to show the connection from flag to another. It might be possible with multiple chains used for rigging but that's not very comfortable to deal with.

Idea: add ID property to each used flag. When a flag is clicked, a PPG opens with a button to show flag connections (one lose chain with many bones, old one is deleted). MT also opens/updates an application (have to look into writing command line apps first). I guess that's necessary to preview (and edit) the complete xml code as this would be to awkward to code directly in MT. The xml file is saved on change events. MT reads that xml when a patrol path flag is selected.

21.06.2016 Yay. Ugly PATR xml editor underway. "Full manual edits" tab already ready... took long enough. Figured out command line part. Next stop MT interaction.


Editing simultaneously multiple ghost quads heights

Onisplit-generated ghost quads vertex ids can differ sometimes. So before changing multiple ghosts in height, their lowest and highest points must be identified.

Display keywords

SetUserKeyword selection(0), "type=akev, id=1" 

for each n in GetUserKeyword(selection(0))
	logmessage n
next

SetValue "Camera.camvis.attrseluserkeywords", False
SetValue "Camera.camvis.attrseluserkeywords", True

xsi_char_save_settings.png


Display flag via ICE

Using a "Simulate Rigid Bodies" ICETree node for this is overkill. But it works and I fail to come up with alternatives, so...

abusing_ICE_for_displaying_a_flag.png 

'### setup flag display by abusing ICE
CreatePrim "Cube", "MeshSurface"
ApplyOp "ICETree", selection(0), siNode, , , 0
AddICENode "$XSI_DSPRESETS\ICENodes\SimulateRigidBodiesNode.Preset", selection(0) & ".polymsh.ICETree"
ConnectICENodes selection(0) & ".polymsh.ICETree.port1", selection(0) & ".polymsh.ICETree.SimulateRigidBodies.simulate"
AddICENode "$XSI_DSPRESETS\ICENodes\IntegerNode.Preset", selection(0) & ".polymsh.ICETree"
ConnectICENodes selection(0) & ".polymsh.ICETree.SimulateRigidBodies.precision", selection(0) & ".polymsh.ICETree.IntegerNode.result"
DisplayPortValues selection(0) & ".polymsh.ICETree.SimulateRigidBodies.precision"
SetValue selection(0) & ".polymsh.ICETree.IntegerNode.value", 7
SaveKey selection(0) & ".polymsh.ICETree.IntegerNode.value", 1

'### check if ICETree op exist on this object
for each oOperator in selection(0).ActivePrimitive.ConstructionHistory
	if oOperator.Name = "ICETree" then
 		got_ICE = 1
  	end if
Next
if got_ICE then
  		logmessage "found ICETree"
else
  		logmessage "couldn't find ICETree"
end if


More wild ideas

Animated reference

Every step on the timeline triggers an event, namely siOnTimeChangeEvent.

It could be used to exchange an the image of an object.

This could be either good for an animated textures (BINA3RAPd*.oni) or for an animated reference for TRAM creation.

Strategy:

  • Making a folder named "Animated_Ref".
  • Drag and drop the folder into Mod Tool.
  • A grid becomes created on a new layer "Animated_Ref". Selection disabled.
  • Mod Tool gets then the first image and looks for the pixels.
  • Based on that pixels the grids becomes deformed to match the image's proportions.
  • Then this "Animated_Ref" grid should be ready for usage.


Similar can be done with the Rotoscope.

Something like that: 

function siOnTimeChangeEvent_OnEvent( in_ctxt )
	' fixed code
	'SetValue "Views.ViewD.RightCamera.camdisp.rotoenable", True
	'SetValue "Views.ViewD.RightCamera.rotoscope.imagename", "Clips.Animated_Ref_N_jpg"

	' flexible code
	' find active viewport, set Rotoscope active and choose image
	'SetValue GetViewCamera(GetViewIndex) & ".camdisp.rotoenable", True
	'SetValue GetViewCamera(GetViewIndex) & ".rotoscope.imagename", "Clips.Animated_Ref_N_jpg"


	current_frame = cstr(in_ctxt.GetAttribute("Frame"))
	if GetValue(GetViewCamera(GetViewIndex) & ".camdisp.rotoenable") = true then
	   ' Animated_Ref_60.jpg ' . becomes _
	   SetValue GetViewCamera(GetViewIndex) & ".rotoscope.imagename", "Clips.Animated_Ref_" & current_frame & "_jpg"      
	end if

	siOnTimeChangeEvent_OnEvent = true
end function


New camera animations

last frame of OBANOutroCam02
camera_OBAN_examination_tn_zpsc8295474.png

There's already an excel macro available but now I want to build cam anims without excel.

The last frame's rotation of level1_Final file OBANOutroCam02 is: -0.0158410165 0.854542 0.0261086561 -0.518483639.

Those quaternions are sorted in X Y Z -W order, so we've to watch out a bit when setting the input for converting them to euler rotations in degrees.

If a newly created object (in this case a cube) shall serve as camera placeholder then the X output has to be multiplied by -1 and the Y output has to be reduced by -180. Those changes must repeated again when exporting the object's rotation to xml.

Update: Sept. 29, 2012

If camera is a real one then the additional X and Y changes aren't necessary. (Primitive > Camera > any should do)

Positions and rotations keyframes can be applied to the camera root object.


Adding multiple textures to level geometry [out-dated]

Material and textures are normally stored right under the object. But AKEV geometry can have more than one texture. Here comes polygon clusters in to play.

An example: 

set oCube = Application.ActiveProject.ActiveScene.Root.AddGeometry("Cube","MeshSurface")
SelectObj "cube", , True
SelectGeometryComponents "cube.poly[LAST]"
CreateCluster

AddToCluster "cube.polymsh.cls.Polygon, cube.poly[2-4]"
RemoveFromCluster "cube.polymsh.cls.Polygon, cube.poly[3]"

The cube has now a polygon cluster. More can be added. Each cluster must get its own material before a texture can be applied to the desired polygons.

[...]


TRBS-fitting TRMA creation

With the following code ...

  • the objects get listed one after another (TRMA-ready)
  • the textures get listed for TRMA and TXMP creation
  • the texture sizes get checked to see if "-large" argument is necessary

One more idea is to add support for reflective textures.


Current status:

hexhound hierarchy

hexhound_hierarchy_tn_zpsd82230ce.png 

if selection.count > 0 then
	' any part could be selected, let's find the root body part
	SelectNeighborObj selection(0), 4
	' get all members including the pelvis
	set bodyparts = selection(0).FindChildren( , , siMeshFamily)
	for each member in bodyparts
		logmessage "object name: " & member.name
		if not typename(member.Material.CurrentImageClip) = "Nothing" then
			logmessage "texture: " & member.Material.CurrentImageClip.source.filename.value
			'logmessage "material: " & member.Material.name
			'logmessage "shader: " & member.Material.shaders(0).name
			logmessage "X: " & member.Material.CurrentImageClip.source.Parameters("XRes").Value
			logmessage "Y: " & member.Material.CurrentImageClip.source.Parameters("YRes").Value
		end if
		logmessage "----------------------------------------------------------"
	next
	logmessage "counted body parts: " & bodyparts.count
else
	logmessage "no object was selected"
end if


Canceled

Reimporting and recovering of original vertex shading on doors, consoles, etc.

Q: Can we reimport and recover such data?
A: XML-based AKEV editing isn't possible. But editing the binary file is possible.
However, OniSplit cannot easily reapply original vertex shading since the used M3GMs have different vertex ids.

Old notes from realization attempt:

  1. import a console into Mod Tool
  2. associate an ID (and everything else necessary to build the "BINACJBOconsole.xml" file later)
  3. apply vertex shading
  4. hit level export button to build the BINACJBOconsole and other xml and eventully all level files as onis
  5. now it would be time for the binary AKEV patching
    1. console id can be used to identify the polygons in AKEV's AGQG instance where the vertex shading is stored (<Colors>)
    2. now we have to hope that onisplit imported the mesh's polygons in the same order as Mod Tool read the polygons
    3. getting the vertex color from Mod Tool: there's already a code piece on MT's page ("[42] get vertex color")


I'm not fluent with binary stuff so I will dump some notes here:


Data table

Reading the AKEV's name table was more or less easy. This time we need to look up the data table.

Three things to take care about: reading hex numbers backwards (as usual in Oni), adding relative offsets to data table's offset (took a pretty while to find that one out*), subtracting 8 bytes for no other reason to get the correct offset.

* Regarding binary information I found THAT page very useful.


What AKEV data do we need? The structure is fixed, so actually only the size or the item count is needed to pin down all colors.

data table offset at 0x20
AGQG offset (A) at 0x94
AGQG size (B) at 0x9C
AGQG items at 0x20 + A + 0x1C - 8
AGQG start at 0x20 + A - 8
AGQG end at 0x20 + A + B - 8


Color data in AGQG

The byte array is like this: BF BF BF FF (191 191 191 255 = grey).

The format is BGRA whereby I couldn't see Alpha to have an effect on the color strength. Maybe it will be different with a texture (not tested yet).


AGQG reader

https://dl.dropboxusercontent.com/u/139715/OniGalore/temp/AKEV_AGQG_reader.txt

output sample: 

' INFO : 000280
' INFO : data table offset: 640
' INFO : 0004A8
' INFO : AGQG table offset: 1192
' INFO : 0001E0
' INFO : AGQG table size: 480
' INFO : ------------------------------
' INFO : AGQG_start: 1824
' INFO : ------------------------------
' INFO : AGQG array size: 8
' INFO : ----------
' INFO : element: 1
' INFO : 00000000
' INFO : 01000000
' INFO : 02000000
' INFO : 03000000
' INFO : 00000000
' INFO : 00000000
' INFO : 00000000
' INFO : 00000000
' INFO : BFBFBFFF (color: 191 191 191 255)
' INFO : BFBFBFFF (color: 191 191 191 255)
' INFO : BFBFBFFF (color: 191 191 191 255)
' INFO : BFBFBFFF (color: 191 191 191 255)
' INFO : 00000804
' INFO : FFFFFFFF (object id: -1)
' INFO : ----------
' INFO : [...]


Complications

M3GM files use triangles while AGQG instances use quads (look, there are always 4 points each with its own 32-bit color slot).

That means Onisplit converts the M3GM triangles into quads on import.

So Mod Tool would need to exactly mimic OniSplit's way turning triangles into quads.

This information is based on following experience:

  • Console0 triangles = 94
  • After triangle-to-quad conversion.
Console0: 47 quads = 38 (main) + 6 (collision box) + 1 + 1 + 1 (light tex quads)
  • AGQG has 47 elements.


Failed on:

  • read the AKEV xml file to get the mesh's vertices/quads (slow and uncool)
  • extract the mesh as dae from AKEV and use that as vertex shading object (reference for M3GM) (faster but uncool because it still requires all consoles, doors, etc.)

AKEV data and extracted console mesh have different polygon IDs and point IDs. Identifying each point by position (taking tolerances into account) and then identifing the quads and triangles would take too long. So, it's still possible but at the moment it's a black hole consuming too much dev time... Darn.

Retrieved from "https://wiki.oni2.net/w/index.php?title=Talk:Mod_Tool/OniTools_addon&oldid=25746"