---

Added by vl 7 months ago

Currently shaders on nel are hardcoded and need to have an opengl and direct3D version of each shader.

The reason is that NeL support old 3d cards that doesn't have shader language.

For example, for the water, there's a direct3d version, an opengl version but also some specific version for 3d card without shader programs.

pro is that it works on old 3d cards

con is that it's a pain to create new shader.

Before Nevrax closed, they worked, in a branch, on a new shader system.

Nevrax made:

• A system to plug a shader to a CMaterial.
• A 3DS script like nl_material.ms to load a shader and setup parameters.
• A system to parse the shader and plug the shader constants to some 3d engine parameters like light positions ...

So you can do some per pixel lightning or special effects on meshes.

pro the system is already up and running so it'll not require lot of work

con seems that it was a bad way to do shader (sfb)

Since we have an opengl and direct3d driver, we need a portable solution.

Perhaps another solution is to use [Cg](https://web.archive.org/web/20090419014148/http://en.wikipedia.org/wiki/Cg_(programming_language%29) that works on both driver

pro perhaps a better standard?

con need lot of code development

Let's talk about alternative, pros and cons, what do you think about all that (since i'm not an expert in this domain, i only tried to write what i understood.

It seems that OGRE and crystal space already use Cg

"0 - Current state" actually works fine, so we don't need to change it for the moment.

"1 - Nevrax System" seems to add dependencies to D3D in NL3D so it's bad for portability. I succeeded to compile it before and it needed some changes but it worked (under windows at least).

"2 - Implement another system" should be perfect, but will need some work.

So I will suggest to keep the actual system (0) while developping system 2 :)

I just discovered COLLADA also supports a cross-plaftorm shaders definition.

http://www.khronos.org/collada/

COLLADA FX

COLLADA FX is the first cross-platform standard shader and effects definition written in XML. It targets high-end systems running OpenGL Shading Language (GLSL) and Cg (HLSL coming), as well as resource-constrained systems (OpenGL ES 1.x profile). It provides:

• Next generation lighting, shading and texturing
• High level effects and shaders
• Support for all shader models (1.x, 2.0, 3.0) under CG and GLSL profiles.

I'll kind of run through this point for point and someone can argue me later. (=

Point 0: Current State

You will forever be writing shaders for multiple systems until you make a revolutionary move to a system like Cg. We looked at writing some shader support specifically for Cg in OpenNeL but we faced a lot of licensing issues. Largely that if you add a proprietary library like Cg into a GPL library you get kicked out of main and shoved into some other repository like non-free or contrib. Or in some cases (Debian) excluded entirely from the packaging systems. For Ryzom, for example, this may not matter but for a FOSS project it does.

If you look at what you stated about Collada you still face one battle or the other: Collada does GLSL which only works for the OpenGL driver which means you need Cg for the DirectX driver.

Today you have to write shaders in assembly which is challenging but not insurmountable if you have someone who knows assembly. The true problem with this low-level approach is that you not only end up with assembly targeting OpenGL and DirectX but potentially (and already in several cases) assembly in either driver targeting specific cards or families of cards.

The other problem is that adding fragment programs requires significant hacking in the rendering pass functionality and writing a vertex program is only slightly easier - write something using CVertexProgram, make sure it is IClassable and then modify the 3DSMAX plugin to know about it so that it can polyptr it into the export. This particular problem could be fixed by making a private method, public.

Point 1: Nevrax System

To say that this system works today is misguided. It compiles today. That essentially is about it. Anything other than a simple flat color programs tend to fail. What this system essentially does is take your shader, parse it and then try to "write" assembly targeting a driver. It fails at this in many many cases. Parsing assembly for OpenGL and then outputting new assembly with constants transposed and commands potentially altered to target a different (DirectX) driver is a really terrible way to address this problem.

This code is also, pardon my language, half-assed. This was a pet-project from someone inside the company and not a concerted initiative to modernize the engine. I think it has some of the basis for a good system but it needs to be steered away from this concept of parsing assembly. In fact the whole concept of writing shaders in a low-level language ended shortly after advanced development on NeL ended (circa 2002.) Everyone pretty much has moved on to GLSL, HLSL and Cg. Some are flirting with high-level shader meta-programming techniques and libraries like Sh to accomplish this.

Point 2: Implement Another System

This is the tactic that I wanted to approach in the OpenNeL community but there are a couple problems. Time and people. I have enough knowledge to write the API and some of the logic around a new system but I do not have the knowledge to integrate it into the NeL 3D engine and drivers. I personally think this is the right approach. If you asked Nevrax/Gameforge/Newowners they may agree with you that Cg is a good idea. It's free, it meets the goals in a single library, etc. I still hesitate to put all of my eggs in the Cg basket. I envisioned a much more robust system similar to the way OGRE and other sophisticated 3D engines accomplish this. You'd have a unified interface to load shaders in and that system would load them from files not code. So the particulars of the implementation to an artist/modeller do not have to be known. This kind of system you also throw assembly by the way side although it is feasible to leave it remain there. In practical implementation you can allow an artist to write their shader in GLSL, HLSL or Cg and specify the constants they need by name which are passed into the program by means of a parameter. No problem-prone parsing is required. This also allows artists to write generic shaders or write directed-shaders - meaning you could do something simple like a glow in Cg but maybe you'd do three shaders for water since you want to eke the most performance out of such a costly program.

Point 3: COLLADA and COLLADA FX

I really like COLLADA and I really want to see COLLADA in the system. After a ton of research and collaboration with some of my friends who are much more savvy with 3D, shaders and in a few cases NeL 3D the problem with COLLADA and COLLADA FX is two fold. COLLADA FX is simply a way to cross-platform represent shaders in a human readable format. Once COLLADA FX loads these shaders in to the system you still have to modify your drivers (I stress plural) to utilize these shaders. Doing COLLADA FX is about half of the work of Point 2, as it eliminates that whole profile/loader system.

But there's a larger problem with implementing COLLADA in NeL and that problem is NeL itself. NeL has two problems in this regard: it was designed 8 years ago and it's written specifically to accept data from 3DSMAX. As I've stated in the IRC channel a couple times this has been a frustration for a number of people who have made attempts to write exporters for other modelers (Maya, XSI and Blender for example) and realistically COLLADA support is not very different from a modeler exporter - you're just eliminating code married to the modeler API. The problem is that 3DSMAX uses interpolation techniques that either no one else uses (TCB, for example) or that have quirks that only exist in 3DSMAX that the Nevrax team had to write around. Writing COLLADA support in would be a significant undertaking as we'd have to re-engineer complex systems, add new interpolation types, new material types and so much more. This cannot be a one-man project.

umm I know this is an old thread, however a multi-platform solution is SDL... Which is heavily developed and used in TONS of open source and Commercial games...

SDL is not a 3D engine (look at : http://www.libsdl.org/cgi/docwiki.cgi/Using_OpenGL_With_SDL). It only supports OpenGL initialization but not Direct3D at all (which can be a real problem for ATI cards).

In all SDL codes with OpenGL, it is using OpenGL standard headers so exactly as NeL does.

:|

datajunkie,

What we're looking for is something more like OGRE's shader system which is language independent and allows selection of register constants at runtime. Here's a document on OGRE's HLSL shaders: http://www.ogre3d.org/wiki/index.php/HLSL

It requires some changes within the system - namely the ability to assign VPs and FPs at a material level (and the underlying classes to support this.) It would also be highly beneficial to be able to save NeL materials independent of their models for this reason as well. We would also have to modify the render pipeline to add additional passes. Right now the passes are hard coded, the pass logic would have to be converted from an if/else statement to a list, vector or possibly deque but some thought must go into how the existing non-standard passes are handled in the new system (e.g. water.)

Basically it's going to require 1-2 skilled 3D programmers with both OpenGL and Direct3D knowledge to accomplish. Unfortunately I'm not that guy and the only one I know doesn't work alone and doesn't really do Direc3D.

I hope this helps explain the situation better...

I am looking into ogre as well for another project outside of Syris V. I am not a 3D guy either and my 3D guy is insanely busy, but I will talk to him about it and see what he thinks...

Going to poke around a bit with Cg in NeL.

http://dl.kaetemi.be/nel/bump.png