Proposal for a graphics pipeline DSL

Geomerics are making a proposal to the UK Technology Stategy Board (TSB) for "disruptive technology" funding to germinate an idea we've been discussing for a long while now. The proposal is a very public one via a youtube video submission, and the "5th judge" will be public feedback! So I'm looking for as much input as possible to improve this idea. Naturally, this is funding proposal, rather than a tech one, so apologies if it's short on tech details.

UPDATE: The video has been submitted. Thanks to all that contributed! Please check it out and leave feedback on the TSB site - you are the 5th judge so your opinion matters!

Here is the final text for the 2 minute video.

Efficient Real-Time Graphics Through A Domain Specific Language

Parallel computing is the next major challenge for software development as chip manufacturing has hit physical limits that require us to go parallel to do more. However, writing parallel software is a hard problem, and requires a new approach.

Computer graphics leads parallel hardware development and is the best known parallel application. But the same hardware and software that drives the graphics in your modern PC is also used by the scientific and medical imaging communities. Advances in graphics frequently have a wider impact in these fields. But despite its apparent suitability, graphics still suffers from a serious programmability gap. There is not yet a good way of driving the hardware effectively. It is widely accepted that without further innovation, faster hardware will not deliver comparatively improved visual quality.

So our goal is to bridge the programmability gap for graphics. By doing so we will tap the unexploited potential of the new power-efficient parallel hardware, providing an important product for the games and graphics industry, while making progress into the wider parallel programming problem.

With full TSB funding, we would draw on our expertise to prototype an alternative graphics pipeline, taking the novel approach of structuring it as a domain specific language (DSL). We will show that new graphics algorithms can be efficiently developed in this language and provide solutions to a range of outstanding problems affecting game and graphics developers that are hard or impossible to realise with existing approaches.

These challenges include:
- higher fidelity images (anti-aliasing)
- cinematic lens effects (bokeh, depth of field)
- complex illumination and shadowing
- semi-transparent materials, such as glass or water
- volumetric rendering, such as fog and smoke

We intend to show a step-change in visual quality, and demonstrate a new parallel programming model with wider applicability. The TSB funding would allow us to build the foundation for a middleware product Geomerics would commercialise in the games and graphics industry. With partial funding we will scale the prototype to deliver a vertical slice focussing on a single rendering challenge.

With its worldwide reputation for game graphics, its close ties with hardware manufacturers and its relationships with academia, Geomerics is ideally placed to carry out this development.

There is some good background material on this topic from this year's Beyond Programmable Shading Course from SIGGRAPH 2010. Of particular relevance is, Johan Andersson's "5 Major Challenges in Interactive Rendering" - another crowd-sourced proposal.

Input and feedback appreciated! If you would prefer not to post publicly, feel free to email me directly (sam.martin@geomerics.com).


Comments

  1. Quote

    I think using a DSL makes a lot of sense. The advantages of a DSL are that an engine can understand the language and re-order the operations according to what best fits the architecture it is running on. The problem with a DSL is that it is hard to get to a point where you can write real software and rely on the DSL system to work. It's very hard to test a new language: for correctness, for performance and for completeness.

    Impact: real-time graphics quality is starting to get stuck. We aren't moving much further, I don't think. I think that there is a problem of getting more programmable hardware, and actually being able to program the hardware. There is a problem that as you add more visual effects, there is a combinatorial explosion, because effects need to work together.

  2. Quote

    Hi Andrew, thanks.

    I agree. To get a DSL to a usable productised form is not a minor hurdle. I believe the best approach is to focus on a limited vertical slice, and expand sideways as you can demonstrate your utility. Allowing incremental adoption will always be a important factor to take into account.

    To pick something tangible, I think OptiX is a good example of the DSL approach. It is effectively a compiler, but delivers a reusable graphics technology that would be very hard to express by any other means.

  3. Quote

    Hi Sam,

    we're doing work with TCD in Dublin using the Halide DSL for image/vision processing leveraging our LLVM backend for the Movidius SHAVE processor and results so far look very encouraging.

    Andrew have and continue to work together on a variety of projects including FP7.

    Perhaps your DSL idea could be the basis for a Vision 2020 proposal uniting computer vision and graphics.

    The big application would be to extract wireframe from real environments in realtime which can be rendered using a GPU making any environment gameable.

    What do you think?

    -David

  4. Quote

    Hi David,

    I have heard of Movidius but I'm not sure what you do exactly! I have your email now, so I'll drop you a line.

    Cheers,
    Sam

Leave a Comment

(required)

(required)

Formatting Your Comment

The following XHTML tags are available for use:

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

URLs are automatically converted to hyperlinks.