Managed DirectX with Win8 Metro Style App

Since my last post is quite old, it's time to give more insight about the latest features included in the recent release of SharpDX 2.0 beta. This release is a major step for SharpDX as it includes lots of new APIs but also provides a preview of developing Managed DirectX from a Windows 8 Metro style application. In the meantime, I have been pretty busy as I got a job in a gamedev company located in Japan that is developing a new rendering engine in C#... which is using SharpDX for its windows rendering, and that's extremely exciting to work on a project like this (and of course rewarding for all the investement done in SharpDX!).

While SharpDX is starting to cover almost the whole DirectX API as well as some new Windows Multimedia API (like WIC), I put some effort, just after the Microsoft Build conference, on providing a preview of using SharpDX from a Win8 Metro style application, which is a fantastic opportunity for a C#/.Net developer to use DirectX from both a Metro style application and Desktop application using the same API.

Finally SharpDX is also going to get some new APIs and some interesting features in the following months, especially in the performance domain, to reduce as much as possible the difference of performance between a native C++ DirectX application and a managed C# DirectX application, that will deserve a post on its own. But lets start with what has been achieved in the latest 2.0 version!

SharpGen a new C# code generation tools from C++

The core changes behind the 2.0 version was the rewrite of the code generator used to generate C# interop code from DirectX/Windows SDK headers, a tool called SharpGen. When I released the first version of SharpDX, I used a handwritten C++ custom parser that was somewhat able to parse almost all DirectX headers, but It was of course a temporary workaround... End of December last year, I was evaluating two options to rewrite the parser stage:

• CLang seemed to be a suitable solution for the future, but at that time, I was not sure that It would be able to parse all windows headers flawlessly (while being able to grab all Microsoft specialized SAL-annotations from their headers in order to extract useful information used by the parser) and It was not really easy to handle (I need to wrap the library into a managed component).
• gccxml is an older project, but is able to parse almost all Microsoft headers with a direct command line and It is quite easy to patch the headers that are not working. Gccxml is generating an xml file, which is pretty easy to parse.

So I made the choice to go with gccxml. Bur this is fortunately not a huge design issue, as it is quite easy in the current system to plug another parser, because the C++ model used by SharpGen is independent from the parser.

The parser was one thing to rewrite in the old system. The other part was to use a new data-driven config files for all mapping rules (used to translate C++ objects to C#). The previous version was quickly developed using hard-coded rules directly written in the C# program. The new version is using simple xml config files to express all the mappings and dependencies. The good thing is that I didn't have to change so many things in the code generator, though lots of work was required to efficiently manage those configs files and their dependencies. You can have a look at the result, for example with SharpDX.DXGI mapping file. The new system has lots of features to manage all the cases that were found during the construction of all these mapping rules. It is also able to efficiently generate a subset of the files, when only parts of the config files are changed and they don't affect other dependent projects (For example, SharpDX.Direct3D11 has some dependencies on D3DCompiler, DXGI and the core SharpDX assembly. If any of the dependent config files are changed, I had to regenerate Direct3D11 as well). All this hard work was clearly done in order to have a system easier to maintain and to fix. The new generator is also able to handle all Windows headers files, including all multimedia files that were not part of the DirectX SDK headers (like WIC).

Thanks to this new tool, It is now possible to integrate in the generated SharpDX assemblies some APIs that were not part of DirectX, but are hightly related to the development of multimedia applications, and that's really a good news for any developer seeking for an unified managed API for all the Windows Graphics and Multimedia APIs.

I do plan also to release SharpGen as an external tool so that you can use it in your own project. It will allow a developer to generate easily interop mapping from C++ headers (with a COM oriented APIs)

Mapping all DirectX APIs

The next step was to write mapping rules and C# extensions for all the remaining APIs. The first version of SharpDX was providing managed APIs for DXGI, Direct3D10, Direct3D11, Direct2D, DirectWrite, DirectSound, XAudio2 but I had to work on older APIs that were more laborious to map like Direct3D9, DirectInput or minor APIs like X3DAudio, XACT3, RawInput, XInput.

I decided also to provide a managed API for WIC (Windows Imaging Component) that is tightly used with Direct2D.

So far, the result is great, except for Direct3D9 that still requires more work, all the DirectX APIs are now provided. SharpDX is also the only managed API to provide an implementation for all the callbacks used in Direct2D/DirectWrite, allowing a full access of these APIs.

I took also the opportunity to rewrite the callback system (C++ is calling back C# objects as they are exposed to the C++ as COM objects). The new system is more efficient and reliable.

Using SharpDX from a Win8 Metro Style Application

A new exciting feature that was really easy to provide is the ability to use SharpDX from a Win8 Metro Style Application. In the meantime, the 2.0 version is also providing an access to the upcoming DirectX11.1 APIs (that includes DXGI 1.2, Direct2D.1 Direct3D11.1, new D3DCompiler, new XAudio2... etc.).

Two samples were ported: one from the Win8 DirectX tutorials that is only clearing the screen. The other is a direct port of SharpDX Direct3D11 MiniCube, a simple application that displays a 3D spinning cube on the screen. You can download the preview archive, compile and run it from a Win8 machine!

In order to be fully compatible with Win8 metro application and to be able to develop application that will get certification on the Microsoft AppStore, some SharpDX internals are being rewritten in order to remove dependencies to legacy Win32 and .Net Framework APIs that are not supported by metro style application. Next version of SharpDX will provide assemblies that will be fully compatible and certified-ready for Win8 Metro style application.

The great thing is that It will be possible to target desktop application and Win8 metro style application using the same managed DirectX API.

You may ask if pure DirectX WinRT components could be generated using SharpDX technology? : in short, yes It is possible (as SharpGen could be modified to generate interop code for other languages), but I don't think this a a good opportunity for .NET developers, as the WinRT system will only be available to pure Win8 Metro style Application. Plus, WinRT has a significant performance cost when you want to have access to static properties/methods (that implies severals method calls : QueryInterface..etc).

Also, I'm not entirely convinced that the new WinRT COM interop projection in .NET is as fast as custom interop used by SharpDX. I didn't have yet the time to write a full benchmark test, but I found some unnecessary generated interop methods by using WinRT components from .NET. So when I will have a chance to install Win8 on a real machine (and not in the VM), I will post a more detailed benchmark about WinRT calling cost from a managed application.

Next?

There are lots of new exciting features that will also be available in next versions of SharpDX, but I can't talk about them yet. In short, SharpDX will get new APIs binding for some Windows multimedia APIs (for example, someone suggested me on twitter to provide the UIAnimation Manager API that I just started today) and will provide some great performance benefits as well.


Also, I have seen a growing interest from XNA developers about SharpDX. It seems that the lack of communication about the future of XNA during and after the Build conference has generated lots of trouble in the XNA community. I don't know if XNA is going to be abandoned by Microsoft. It may be probably a bit to early to anticipate this. On the other hand, I wrote a couple of months ago a significant part of the Graphics layer of XNA on top of SharpDX, using internally Direct3D11, but I didn't release it, as it was not yet in a usable state. After the final 2.0 version of SharpDX will be released, I will have a look to see if this framework could be released as-is, in a state that would require some work from the SharpDX community.

Finally, I'm receiving some great feedbacks from developers using SharpDX in some commercial projects and that's really great! I would be glad to hear more news from users, even If it can't be public, that's extremely interesting to understand SharpDX usages and improve the API experience as well.

Stay tuned!