A new managed .NET/C# Direct3D 11 API generated from DirectX SDK headers

I have been quite busy since the end of august, personally because I'm proud to announce the birth of my daughter! (and his older brother, is somewhat, asking a lot more attention since ;) ) and also, working hard on an exciting new project based on .NET and Direct3D.

What is it? Yet Another Triangle App? Nope, this is in fact an entirely new .NET API for Direct3D11, DXGI, D3DCompiler that is fully managed without using any mixed assemblies C++/CLI but having similar performance than a true C++/CLI API (like SlimDX). But the main characteristics and most exciting thing about this new wrapper is that the whole code marshal/interop is fully generated from the DirectX SDK headers, including the MSDN documentation.

The current key features and benefits of this approach are:

• API is generated from DirectX SDK headers : the mapping is able to perform "complex transformation", extracting all relevant information like enumerations, structures, interfaces, functions, macro definitions, guids from the C++ source headers. For example, the mapping process is able to generated properties for interfaces or inner group interface like the one you have in SlimDX : meaning that instead of having a "device.IASetInputLayout" you are able to write "device.InputAssembler.InputLayout = ...".
• Full support of Direct3D 11, DXGI 1.0/1.1, D3DCompiler API : Due to the whole auto-generated process, the actual coverage is 100%. Although, I have limited the generated code to those library but that could be extended to others API quite easily (like XAudio2, Direct2D, DirectWrite... etc.).
• Pure managed .NET API : assemblies are compiled with AnyCpu target. You can run your code on a x64 or a x86 machine with the same assemblies. 
• API Extensibility The generated code is in C#, all the types are marked "partial" and are easily extensible to provide new helpers method. The code generator is able to hide some methods/types internally in order to use them in helper methods and to hide them from the public api.
• C++/CLI Speed : the framework is using a genuine way to avoid any C++/CLI while still achieving comparable performance.
• Separate assemblies : a core assembly containing common classes and an assembly for each subgroup API (Direct3D, DXGI, D3DCompiler)
• Lightweight assemblies : generated assemblies are lightweight, 300Ko in total, 70Ko compressed in an archive (similar assemblies in C++/CLI would be closer to 1Mo, one for each architecture, and depend from MSVCRT10)
• API naming convention very close to SlimDX API (To make it 100% equals would just require to specify the correct mapping names while generating the code)
• Raw DirectX object life management : No overhead of ObjectTable or RCW mechanism, the API is using direct native management with classic COM method "Release". Currently, instead of calling Dispose, you should call Release (and call AddRef if you are duplicating references, like in C++). I might evaluate how to safely integrate Dispose method call. 
• Easily obfuscatable : Due to the fact the framework is not using any mixed assemblies
• DirectX SDK Documentation integrated in the .NET xml comments : The whole API is also generated with the MSDN documentation. Meaning that you have exactly the same documentation for DirectX and for this API (this is working even for method parameters, remarks, enum items...etc.). Reference to other types inside the documentation are correctly linked to the .NET API. 
• Prototype for a partial support of the Effects11 API in full managed .NET.

If you have been working with SlimDX, some of the features here could sound familiar and you may wonder why another .DirectX NET API while there is a great project like SlimDX? Before going further in the detail of this wrapper and how things are working in the background, I'm going to explain why this wrapper could be interesting.

I'm also currently not in the position to release it for the reason that I don't want to compete with SlimDX. I want to see if SlimDX Team would be interested to work together with this system, a kind of joint-venture. There are still lots of things to do, improving the mapping, making it more reliable (the whole code here has been written in a urge since one month...) but I strongly believe that this could be a good starting point to SlimDX 2, but I might be wrong... also, SlimDX could think about another road map... So this is a message to the SlimDX Team : Promit, Josh, Mike, I would be glad to hear some comments from you about this wrapper (and if you want, I could send you the generated API so that you could look at it and test it!)

[Updated 30 November 2010]
This wrapper is now available from SharpDX. Check this post.
[/Updated]

This post is going to be quite long, so if you are not interested by all the internals, you could jump to the sample code at the end.

NShader 1.1, hlsl, glsl, cg syntax coloring for Visual Studio 2008 & 2010

I have recently released NShader 1.1 which adds support for Visual Studio 2010 as well as bugfixes for hlsl/glsl syntax highlighting.

While this plugin is quite cool to add a basic syntax highlighting for shader languages, It lacks intellisense/completion/error markers to improve the editor experience. I didn't have time to add such a functionality in this release as... I don't really have too much time dedicated to this project... and well, I have so much to learn from effectively practicing a lot more shader languages that I'm fine with this basic syntax highlighting! ;) Is it a huge task to add intellisense? It depends, but concretely, I need to implement for each shading language a full grammar/lexer parser in order to provide a reliable intellisense. Of course, a very basic intellisense would be feasible without this, but I would rather not to use an annoying/unreliable intellisense popup.

Building .fx hlsl files with a Custom Build Rule under Visual Studio 2008

It's sometimes convenient to be able to compile directly a .fx file under a VS C++ project. NShader provides a nice way to highlight your code, but it's not able (yet, not sure I will update it up to this feature) to compile anything or to display syntax errors.

Hopefully, there is a simple workaround, avoiding to develop your own custom building task : using a custom build rule. Unfortunately, I was able to find one a while ago on Internet, but I lost it, and if I remember, it was lacking some features (like showing all available profiles)... So I decided yesterday to implement my own building rules files, leveraging on the existing .asm rules files that is shipped with Visual Studio, and It was surprisingly easy to write it.

To add this custom building rules to VS, follow these instructions:

AsmHighlighter Part II : Automatic x86 asm instruction size evaluation

Following the previous post about AsmHighlighter, I have just released a new version 1.1 of AsmHighlighter that evaluates x86 ASM instruction size for each instructions in your code. The instruction size is inserted in the comment next to the instruction.

When you are coding 4k or 64k (partially) demos in x86 assembler with size optimization in mind, it's sometimes useful to check the instruction size (as it is sometimes not so obvious, x86 has a lot of weird rules for their instruction size, particularly when using eax register on source/destination, or using ebx without an offset...etc).

This feature is available when you perform a formating of your document (Menu Edit/Advanced/Format document or selection, or Ctrl+k, Ctrl+d). Here is a screenshot result:

The magic behind it is that i have discovered a cool managed library called : Fasm Managed from Shynd's. This is a managed .NET dll that can perform dynamic x86 ASM compilation inside a .NET application. The code is simple as below:

byte[] buffer = ManagedFasm.Assemble("mov eax, 0");

Thanks to this great module, it was quite easy to integrate a fast evaluation of a x86 instruction size, although this evaluation is done at the line level, so the evaluation doesn't have any information on labels, defines...etc. But i have integrated a simple (but not perfect) workaround. Also, because this library is using FASM as a compiler, i had to perform some "normalization" on the code (that is mainly targeting MASM syntax), but it's working quite well!


With this instruction size feature, AsmHighlighter is becoming a bit more related to 4k-64k intros coding! ;)

Adding asm x86 Syntax Highlighting to Visual Studio 2008

While i use to develop some part of 4k's intro code in x86 assembler, it is quite annoying to switch from Visual Studio to an external editor in order to have a correct syntax highlighting. I like the principle of an Integrated Development Environment, and VS is a great tool for that. Based on my previous experience on NShader, i have decided to implement a syntax highlighter for x86 ASM... and thanks to NShader's work, it was really easy to code it! For anyone looking for a place to start when developing a LanguageService Package for Visual Studio, this project is easy to follow and gives a basic implementation for syntax highlighting.

So, i have just released AsmHighlighter under codeplex. This is an AddIn for Visual Studio 2008 (it won't work for Express Editions, as they don't allow communities addIns) that enables syntax highlighting for x86 ASM language. MASM syntax is mainly supported and NASM syntax is partially supported. I have no more reason to switch to an external editor! Cool!

Apart from that, the nice thing with MASM is that it is possible to debug directly under Visual Studio, making a mixed C-ASM project much easier to develop (VS is adding building rules for ASM files automatically to your project). On the other hand, i was not able to debug with NASM as nasm cannot generate debug information under the win32 platform...

I also found a cool project called JWasm developed by Japheth : JWasm is a MASM v6 compatible assembler (with v8 bug fixes and improvements)  that is able to output win32,win64 as well as linux elf object files. Quite interesting when you still want to develop with a MASM syntax and be able to export your work under linux without moving to another assembler! Japtheth even provides JWasm.rules files for an integration under Visual Studio. It's even possible to debug JWasm output objects files under VS, that's great!

Now, you may have noticed that all of this is not strictly related to 4k-64k intros coding, but well, you can consider AsmHighlighter as a productivity tool for 4k-64k development! ;)

Coding a 4k or 64K intro needs tools!

What makes a 4k or a 64k so special compares to others demo categories? Specialized tools of course!

There are some obvious general remarks we may list:

• The smaller the intro, the more specialized tools you will need (eg. crinkler, 4klang...). The bigger the demo (>5Mo), the more you will be able to use standardized tools (Zip, Blender, Renoise, Fmod...).
  
• The smaller the intro (<1k), the less specialized tools are released or even developed.
• The smaller the intro, the more graphists are being excluded from demo development compare to musicians that can still work on a music even in a 1k intro (see Untraceable).