Wednesday, August 25, 2010

Import and Export 3D Collada files with C#/.NET

Looking at what kind of 3D file format I could work with, I know that Collada is a well established format, supported by several 3D modeling tools, with a public specification and a XML/Schema grammar description, very versatile - and thus very verbose. For the last years, I saw a couple of articles on, for example, "how to import them in the XNA content pipeline" or about Skinning Animation with Collada and XNA, with some brute force code, using DOM or XPath to navigate around the Collada elements.

Now, looking at the opportunity to use this format and to build a small 3D demo framework in C# around SlimDx, I tried to find a full implementation of a Collada loader, derived from the xsd official specification... but was disappointed to learn that most of the attempts failed to use the specification with an automatic tool like xsd.exe from Microsoft. If you don't know what's xsd.exe, It's simply a tool to work with XML schemas, generate schemas from a DLL assembly, generate C# classes from a xsd schema...etc, very useful when you want to use directly from the code an object model described in xsd. I will explain later why this is more convenient to use it, and what you can do with it that you cannot achieve with the same efficiency compare to raw DOM/Xpath access.

I had already used xsd tool in the past for NRenoiseTools project and found it quite powerful and simple, and was finally quite happy with it... But why the Collada xsd was not working with this tool?


Patching the Collada xsd

Firstly, I have downloaded the Collada xsd spec from Kronos group and ran it through the tool... too bad, there was indeed an error preventing xsd to work on it
Error: Error generating classes for schema 'COLLADASchema_141'.
- Group 'glsl_param_type' from targetNamespace='http://www.collada.org/2005/11
/COLLADASchema' has invalid definition: Circular group reference.

This error was quite old and got even a bug submitted to connect "xsd.exe fails with COLLADA schema. Prints circular reference problem". Well the problem is that looking more deeply at the xsd schema, the glsl_param_type doesn't make any circular group reference... weird...

Anyway, because this was just an error on the GLSL profile part of Collada spec, I removed this part, as this is not so much used... and did the same for CG and GLES profiles that had the same error.

Bingo! Xsd.exe tool was able to generate a -large - C# source file. I found it so easy that I was wondering why they had so much pain with it in the past? Well, running a simple program to load a sample DAE collada files... and got a deep exception :

Member 'Text' cannot be encoded using the XmlText attribute

A few internet click away, I found exactly a guy having the same error... from the code:
/// <remarks/>
[System.Xml.Serialization.XmlTextAttribute()]
public double[] Text {
    get {
        return this.textField;
    }
    set {
        this.textField = value;
    }
}
XmlTextAttribute specify that the "Text" property should be serialized inside the content of the xml element... but unfortunately, the XmlText attribute doesn't work on arrays of primitives!

Someone suggested him several options, and the simplest among them was to use a simple string to serialize the content instead of using an array... This is a quite common trick if you are familiar with xml serializing in .NET (and also with WCF DataContract xml serialization from .NET). So I went this way... It was quite easy, because the file had less than 10 occurrences to patch, so I patched them manually... with the kind of following code:
/// <remarks />
[XmlText]
public string _Text_
{
    get { return COLLADA.ConvertFromArray(Values); }

    set { Values = COLLADA.ConvertDoubleArray(value); }
}

/// <remarks />
[XmlIgnore]
public double[] Values
{
    get { return textField; }
    set { textField = value; }
}
I put a XmlIgnore on the renamed "Values" property that use the double[] and add a string property that performs a two-way conversion to that values (while adding the ConvertFromArray and ConvertDoubleArray functions at the end of the xsd generated file.

And... It was fully working!

Using Collada model from C#

With the generated classes, this is much easier to safely read the document, to access collada elements, having intellisense completion to help you on this laborious task. I have also added just 2 methods to load and save directly dae files from a stream or a file. The code iterating on Collada elements is something like (dummy code):
// Load the Collada model
COLLADA model = COLLADA.Load(inputFileName);

// Iterate on libraries
foreach (var item in model.Items)
{
    var geometries = item as library_geometries;
    if (geometries== null)
    continue;
    
    // Iterate on geomerty in library_geometries 
    foreach (var geom in geometries.geometry)
    {
        var mesh = geom.Item as mesh;
        if (mesh == null)
        continue;
        
        // Dump source[] for geom
        foreach (var source in mesh.source)
        {
            var float_array = source.Item as float_array;
            if (float_array == null)
                continue;
        
            Console.Write("Geometry {0} source {1} : ",geom.id, source.id);
            foreach (var mesh_source_value in float_array.Values)
                Console.Write("{0} ",mesh_source_value);
            Console.WriteLine();
        }
    
        // Dump Items[] for geom
        foreach (var meshItem in mesh.Items)
        {
        
            if (meshItem is vertices)
            {
                var vertices = meshItem as vertices;
                var inputs = vertices.input;
                foreach (var input in inputs)
                    Console.WriteLine("\t Semantic {0} Source {1}", input.semantic, input.source);                                
            }
            else if (meshItem is triangles)
            {
                var triangles = meshItem as triangles;
                var inputs = triangles.input;
                foreach (var input in inputs)
                    Console.WriteLine("\t Semantic {0} Source {1} Offset {2}",     input.semantic, input.source, input.offset);
                Console.WriteLine("\t Indices {0}", triangles.p);
            }
        }
    }
}

// Save the model
model.Save(inputFileName + ".test.dae");

One thing that could be of an interest, is that not only you can easily load a Collada dae file... but you can export them as well! I did a couple of experiment to verify that importing and exporting a Collada file is producing the same file, and It seems to work like a charm... meaning that if you want to produce some procedural Collada models to load them back in a 3D modeling tool, this is quite straightforward! But anyway, my main concern was to have a solid Collada loader that is compliant with the spec and performs most of the tedious fields conversion for me.

Of course, having such a loader in C# is just a very small part of the whole picture in order to create a full importer supporting most of the Collada features for a custom renderer... but that's probably the less exciting part of developing such an importer, so having this C# Collada model will be probably helpful.



Note: You can download the C# Collada model here. This is only a single C# source file that you can add directly to your project!

The model is stored inside the namespace Collada141 (in order to support multiple incompatible version of the Collada spec), and the root class (as specified in the xsd) is the COLLADA class, which contains also the two added Load/Save methods.

Also, a nice thing about the generated model from xsd.exe is that it allows you to extend the object model methods outside the csharp file. All the classes are declared partial, so It's quite easy to add some helpers method directly inside the Collada object model without touching directly the generated file.

Let me know if you are using it!

14 comments:

  1. Wow, thanks a lot! I've been dealing with this exact same scenario. I was having trouble with the XmlText attribute, but after applying the technique you used my code worked! This was much appreciated. :)

    ReplyDelete
  2. Well, I haven't quite got my hands dirty with it yet, but if all goes well I plan on using this in a prototyping tool I'm developing.

    Cheers!

    ReplyDelete
  3. Well, first implementation of exporter written. Thanks again.

    ReplyDelete
  4. Hi. I've been using the source file you uploaded in my exporter and it's been working great until I tried using float3 and float4 in my effect parameters. According to the documentation this should work, but it's crashing when I try to load a file with those types as effect parameters.

    I tried posting part of the dae file here, but it won't let me use non html tags in my reply.

    I'm not sure about saving because I don't know what type to pass in. In common_newparam_type where it defines the types allowed for Item, it has double for all of float2, float3 and float4. That looks wrong, but I have no idea what to patch it with.

    I tried running the schema through xsd.exe myself and eventually got it to build the cs file, but it still had the same errors and I may have removed a bit more of it than necessary.

    ReplyDelete
  5. @Andree, indeed, the mapping for float2/float3/float4 should be adjusted manually.
    Do you have a sample of the part where the float3/float4 appears and how they are formatted? I could then probably give you a patch.

    ReplyDelete
  6. This is where I'm trying to define the float3 parameter. I copied this from the newparam examples in http://www.khronos.org/files/collada_spec_1_5.pdf

    <profile_COMMON>
    <newparam sid="myDiffuseColor">
    <float3> 0.2 0.56 0.35 </float3>
    </newparam>
    <technique sid="T1">
    <lambert>
    <emission><color>1.0 0.0 0.0 1.0</color></emission>
    <ambient><color>1.0 0.0 0.0 1.0</color></ambient>
    <diffuse><param ref="myDiffuseColor"/></diffuse>
    <reflective><color>1.0 1.0 1.0 1.0</color></reflective>
    <reflectivity><float>0.5</float></reflectivity>
    <transparent><color>0.0 0.0 1.0 1.0</color></transparent>
    <transparency><float>1.0</float></transparency>
    </lambert>
    </technique>
    </profile_COMMON>

    ReplyDelete
  7. Actually it was from the example in lambert section.

    ReplyDelete
  8. Nevermind, I figured it out. I noticed the common_color_or_texture_typeColor class behaved exactly the way I expected float4 to, so all I needed to do was copy that and modify the Item field in common_newparam_type to say [XmlElement("float4", typeof (float4))].

    ReplyDelete
  9. Great, this is exactly the way to fix it.

    ReplyDelete
  10. Hi, I have been using this and it's been very helpful. Thanks very much for posting it. :) I think I may have found/fixed a small problem: I noticed that my Name_array's were coming through empty. I compared float_array with Name_array, and tried changing [XmlElement("Name")] to [XmlText] on the Name_array._Text_ property. This seems to have fixed the problem.

    ReplyDelete
  11. HI,
    I would to test the code you have posted but the link is brocken. Can you post it again?

    Thanks for your help.

    ReplyDelete
  12. Anybody knows if the source is available from somewhere else? The link is dead and the author doesn't seem to be responding.

    ReplyDelete