AE: Settings 16-bit in the Project Settings vs. Render Settings

[Aaron Scott 0]

I have a Render Settings template that has "Color Depth" set to "16 bits per channel" (as opposed to "Current Settings"). I work with my project color settings set for 8-bit, and when I'm done, render using the 16-bit module.

 

My understanding is that, if the color depth is set by the render setting, it overrides the project settings. The AE docs (hey, Todd!) say this:

 

"You can specify a color depth for each render item, which overrides the project color depth when rendering for final output. You can also specify the color depth to use for each output item in the output module settings."

 

I'm having people tell me, though, that setting your project as 8-bit in your Project Settings and then setting it as 16-bit in your Render Settings means that your project is actually getting rendered 8-bit, and then put into 16-bit files. This makes no sense to me whatsoever, but I'm being told that a 16-bit project setting / 16-bit render setting is producing considerably less banding than an 8-bit project / 16-bit render.

 

What's going on here? Is one, or both of us, insane?

[kitkats 0]

I believe it's a bit like scaling up an image in photoshop using 'nearest neighbour' - your output will technically have higher bit depth and colour accuracy but you wouldn't notice it because the values will all be multiples (scaled up) of the 8 bit values. I think the render settings override project settings for the output file but not for the calculations during rendering. I think... (?)

[Aaron Scott 0]

I believe it's a bit like scaling up an image in photoshop using 'nearest neighbour' - your output will technically have higher bit depth and colour accuracy but you wouldn't notice it because the values will all be multiples (scaled up) of the 8 bit values. I think the render settings override project settings for the output file but not for the calculations during rendering. I think... (?)

 

Yeah, that's what I'm being told. But that seems really, really wrong to me -- unless I'm totally interpreting the documentation wrong, it should be rendering at 16-bit, not just containering an 8-bit render in a 16-bit file. Seems like there's a lot of confusion over this.

[nog 0]

It's not wrong. The project settings determine what bit depth the project is in. The render settings determine what kind of file you render to. So for example, I often work in 16 bit but then render to an 8 bit file. That way all of the compositing and effects are done in 16 bit to reduce banding, but the final render is output to a regular 8 bit file. And of course that setup means that you can do what you've been doing and render an 8 bit project into a 16 bit file, even though there's no reason you would ever really want to.

 

edit: I see what you mean now. "overrides the project color depth when rendering for final output" is really confusing because it makes it sound like it should work the way you're describing it.

 

I just tested it though, and if you do what you're saying it even gives you an error message "output module depth exceeds project color depth" or something like that.

Edited May 10, 2011 by nog

[Aaron Scott 0]

I just tested it though, and if you do what you're saying it even gives you an error message "output module depth exceeds project color depth" or something like that.

 

I've done this hundreds of times with ProRes 4444, which is 16-bit only. If Project Settings is set to 8-bit and Render Settings is set to 16-bit, I get a 16-bit ProRes. At no point is there a warning.

 

So, just so we're perfectly clear, this is our best guess:

 

Your bit depth as set in Project Settings / at the bottom of your Project panel is the bit depth anything in your Render Queue will render at.

 

The bit depth as set in your Render Settings, despite being called an "override" for the Project bit depth in the docs, doesn't actually change the bit depth your project renders at, but rather does a dirty upconvert / downconvert to a given bit depth after the render has already taken place.

 

Is this correct? If I chant his name three times, will Todd appear from nowhere to clarify this once and for all?

[Mylenium 0]

I'm having people tell me, though, that setting your project as 8-bit in your Project Settings and then setting it as 16-bit in your Render Settings means that your project is actually getting rendered 8-bit

 

It isn't. It switches the whole render pipeline to genuine 16bit and that, I believe, answers your questions. Everything else is nonsense. The only real question after that becomes whether you could actually verify the difference in a technical sense - even in 16bit, many times color values will be identical to the 8bit ones. Just ramping up to 16bit doesn't necessarily always exhaust the added headroom/ dynamic range...

 

Mylenium

[Mylenium 0]

I just tested it though, and if you do what you're saying it even gives you an error message "output module depth exceeds project color depth" or something like that.

 

I think you're confusing things. It only throws this message, if you render e.g. from an 8bit project to a 16bit file format or something similar >>>without<<< enabling the bit-depth override, which is not what Aaron does.

 

Mylenium

[Aaron Scott 0]

Empirical testing to the rescue!

 

I rendered a simple ramp at four different settings, bringing the results into a 16-bit PS document and cranking the contrast:

 

 

So your color depth is, in fact, determined by your Project Settings, and the AE documentation is wrong when it says it "overrides the project color depth when rendering".

 

I'm honestly surprised. This means that I've been rendering a lot of stuff 8-bit when I thought I was rendering 16. Well, that explains the banding issues I've been seeing.

Edited May 10, 2011 by Aaron Scott

[Mylenium 0]

I'm honestly surprised.

 

You shouldn't have to be. Seriously, this should work and is easily verifiable when rendering stuff like a glow in a different bit depth, in particular rendering anything designed in a 32bit project to lower bit-depths. Even the comp preview will reflect this during rendering. And I can even easily counter your test - rendering an 8bit project to 16bit by overriding the bit-depth in the render settings to a 16bit PSD shows no banding at all when applying a curve adjustment in PS to solarize it. Furthermore, the same 8bit project rendered in 16bit, but saved to a 8bit PSD shows a distinctively different dithering pattern. So AE correctly renders at the bit-depth defined in the RQ settings. I really think you are missing a critical step somewhere.

 

Mylenium

[Aaron Scott 0]

You shouldn't have to be. Seriously, this should work and is easily verifiable when rendering stuff like a glow in a different bit depth, in particular rendering anything designed in a 32bit project to lower bit-depths. Even the comp preview will reflect this during rendering. And I can even easily counter your test - rendering an 8bit project to 16bit by overriding the bit-depth in the render settings to a 16bit PSD shows no banding at all when applying a curve adjustment in PS to solarize it. Furthermore, the same 8bit project rendered in 16bit, but saved to a 8bit PSD shows a distinctively different dithering pattern. So AE correctly renders at the bit-depth defined in the RQ settings. I really think you are missing a critical step somewhere.

 

Then... what could I possibly be missing? This is on a fresh install of CS5.5. Literally all I'm doing:

• Setting the project to 8-bit color depth.
  
• Making a new comp.
  
• Making a ramp.
  
• Adding that comp to the render queue.
  
• Setting the color depth to 16-bit in Render Settings.
  
• Rendering.
  

The result is a 16-bit file that has identical banding to an 8-bit render. If I do the exact same thing but set the project setting to 16-bit before rendering, I get a 16-bit file with no banding.

 

Could it be a 5.5 bug?

[Mylenium 0]

Then... what could I possibly be missing? This is on a fresh install of CS5.5. Literally all I'm doing:

• Setting the project to 8-bit color depth.
  
• Making a new comp.
  
• Making a ramp.
  
• Adding that comp to the render queue.
  
• Setting the color depth to 16-bit in Render Settings.
  
• Rendering.
  

The result is a 16-bit file that has identical banding to an 8-bit render. If I do the exact same thing but set the project setting to 16-bit before rendering, I get a 16-bit file with no banding.

 

Could it be a 5.5 bug?

 

But do you actually set the colors to "trillions" in the output module settings? This is not automatically done because these settings are by default derived from the project settings, which naturally stick at 8bit and thus "millions".

 

Mylenium

[Aaron Scott 0]

But do you actually set the colors to "trillions" in the output module settings? This is not automatically done because these settings are by default derived from the project settings, which naturally stick at 8bit and thus "millions".

 

The tests were all rendered at TIFFs, the top two set at "trillions" and the bottom two at "millions". Opening the files in Photoshop confirms that the 8-bit project settings / 16-bit render settings render TIFF (top-left) is 16-bit.

[nog 0]

Yep, Aaron, that's exactly what I tried. Is there some "bit-depth override" checkbox I'm missing?

[Aaron Scott 0]

Okay, here's a test case.

 

I made an AE file. All it has is a high-contrast ramp, the render queue set to output a PNG (with "trillions of colors").

 

The project bit depth is 8-bit:

 

 

The render settings bit depth is 16-bit:

 

 

According to Mylenium, as well as my common sense, this should be the result:

 

 

That's what the comp looks like when rendered 16-bit. But this is what I get (this is the actual PNGs outputted by After Effects):

 

 

It's the ramp, rendered 8-bit, inside of a 16-bit file. The ramp was not rendered 16-bit.

 

If you render the exact same file, but set the project bit depth to 16-bit, you get that top PNG -- properly 16-bit.

 

You can download the test case here: CS5.5 and CS5

Edited May 10, 2011 by Aaron Scott

[nog 0]

Oh wait, I was missing the "color depth" pulldown and just changing the output file format. Your test file works fine for me. Mylenium is right, as expected.

[Mylenium 0]

If there is any bug, it's Mac-specific. I can produce the expected result in anything from CS3 to CS5.5 here and I would bet I'd even get it right on my AE 6.5 and AE 7 installs at work...

 

Mylenium

[Aaron Scott 0]

Damn. I can absolutely, completely confirm the behavior as I'm seeing it on multiple Mac Pros, running CS5.5 on OS 10.6.7. I didn't test it in CS5, just CS5.5. I'll try CS5 in the morning to see if this is a new bug or not.

[levante 0]

Damn. I can absolutely, completely confirm the behavior as I'm seeing it on multiple Mac Pros, running CS5.5 on OS 10.6.7. I didn't test it in CS5, just CS5.5. I'll try CS5 in the morning to see if this is a new bug or not.

A few month ago i filed a bug report to Adobe about this behaviour (in CS5), seems like they didn't really care...

[Aaron Scott 0]

A few month ago i filed a bug report to Adobe about this behaviour (in CS5), seems like they didn't really care...

 

They're looking into it now. Looks like the bug is related in some way to multiprocessing, but so far they've never been able to reproduce it on their end.

[levante 0]

They're looking into it now. Looks like the bug is related in some way to multiprocessing, but so far they've never been able to reproduce it on their end.

good to know.