Feeding Y'CbCr ("YUV"/"YV12") data to TMPGEnc

Archived from groups: rec.video.desktop (More info?)

Wilbert Dijkhof wrote:

>> Does TMPGEnc support encoding Y'CbCr ("YUV" ) data "as is", or
>> does it _always_ internally convert everything to R'G'B' space,
>> no matter what, before finally encoding to Y'CbCr format?

> It always converts it to RGB24 (by some external codec when
> necessary).

Thank you. This is what I suspected, though it was initially
hard to believe it.

Is there any sensible explanation as to why the author(s) of
TMPGEnc feel it is necessary to convert Y'CbCr data to R'G'B'
and then back to Y'CbCr again, instead of accepting the Y'CbCr
data "as is"?

Are there any decent MPEG-2 encoders out there which do not
unnecessarily muck about with the colorspace?

--
znark

Archived from groups: rec.video.desktop (More info?)

Wilbert Dijkhof wrote:

>> Is there any sensible explanation as to why the author(s) of
>> TMPGEnc feel it is necessary to convert Y'CbCr data to R'G'B'
>> and then back to Y'CbCr again, instead of accepting the Y'CbCr
>> data "as is"?

> I guess editing in RGB is easier than editing in Y'CbCr.
> But, I don't know whether that's the main reason.

It is true that it is easier to do _some_ filtering and color
correction operations in the R'G'B' colorspace. However, some
other things and filters are much easier to do in the Y'CbCr
colorspace, so it cannot really be said that using R'G'B'
would have a clear advantage over anything.

Moreover, providing filtering (or color correction) functions
should not even be the primary concern of an MPEG-2 encoder.
The primary concern of an MPEG-2 encoder should be encoding
data to the MPEG-2 format, which - out of necessity - involves
converting it to the Y'CbCr colorspace at the end of the chain,
since the MPEG-2 format mandates that the data be stored in
the Y'CbCr format. Everything else (noise filtering, color
correction etc.) is just auxiliary fluff and extra in an
MPEG-2 encoder - these things can be better handled in separate
programs, anyway, before feeding the source video data to the
encoder.

Colorspace conversion is, of course, needed if the source
material is not already in Y'CbCr format. However, these
days, this is usually never the case. Y'CbCr - and not R'G'B'
- is the most common format in digital video. DV is based
on Y'CbCr, M-JPEG is based on Y'CbCr, MPEG-2 is based on
Y'CbCr, DivX/Xvid/MPEG-4 is based on Y'CbCr, and practically
all capture cards (analogue tv tuner cards etc.) capture in
the Y'CbCr format. Forcing the data to R'G'B' and then back
to Y'CbCr again "just because" does not make any sense.

It is hard to see why the programmers chose not to support
direct Y'CbCr input (YUY2, YV12 or both) in TMPGEnc - it
would have been easier for them, too. (It also should
not be a big deal to add this support in the forthcoming
versions, if they just wanted to.)

>> Are there any decent MPEG-2 encoders out there which do
>> not unnecessarily muck about with the colorspace?

> There's CCE which converts to YUY2 and back. It should be
> faster.

I have heard a lot of good things about Cinemacraft. I think
I'll have to try out the trial versions - though I could only
afford the basic version; the pro ones are definitely too
expensive. Hopefully it is not too limited.

> Since a few weeks/months there is also Mencoder and QuEnc
> (which can accept avs scripts) where there's no conversion
> at all. See Doom9's forum (or kvcd.net) for details.

Interesting, thanks for the tips!

--
znark

Archived from groups: rec.video.desktop (More info?)

On a sunny day (Wed, 19 May 2004 18:03:25 +0300) it happened "Jukka Aho"
<jukka.aho@iki.fi> wrote in <ZEKqc.2095$ZO4.874@reader1.news.jippii.net>:

> Since a few weeks/months there is also Mencoder and QuEnc
>> (which can accept avs scripts) where there's no conversion
>> at all. See Doom9's forum (or kvcd.net) for details.
>
>Interesting, thanks for the tips!
>
>--
>znark
>
You should all go for Linux and mjpegtools.
YUV in, resizer filters... and free.
mencoder (as from mplayer) in Linux has been around for at least 1.5 years.
JP

Archived from groups: rec.video.desktop (More info?)

On a sunny day (Wed, 19 May 2004 18:07:15 +0300) it happened "Jukka Aho"
<jukka.aho@iki.fi> wrote in <yIKqc.2097$8U4.484@reader1.news.jippii.net>:

>Jukka Aho wrote:
>
>> Forcing the data to R'G'B' and then back to Y'CbCr again
>> "just because" does not make any sense.
>
>Forgot to add this (in case it is not obvous): "...since
>each conversion from Y'CbCr to R'G'B' or back is a lossy
>operation, which degrades the image quality."

Because it is a simple mathematical formula, done forward and reverse.
You can do it for each pixel!
And that is what one would do of cause internally in a program, if color
processing in UV was needed.
See example (not 100 % identical color standards, I just pasted some code).


/* Example of forward in double */

double cr, cg, cb, cu, cv;

/* One pixel, colors in R, G, B */

/* color standard */
/* Y spec */
cr = 0.3;
cg = 0.59;
cb = 0.11;

/* U spec */
cu = .5 / (1.0 - cb);

/* V spec */
cv = .5 / (1.0 - cr);

Y = cr * R + cg * G + cb * B;

/* the 128 is so as to use 8 bits and unsigned */
U = cu * (B - y) + 128;

V = cv * (R - y) + 128;

/* One pixel, colors now in Y, U, V */



/* Example of reverse in 8 bits integer calculation */
/* note color standard is a bit different in this example */

#define LIMIT(x) ((((x)>0xffffff)?0xff0000((x)<=0xffff)?0x)&0xff0000))>>16)

cy = ( (0xff & * Y) - 16);
cy *= 76310;

/* we remove the 128 here, to get signed again */
cu = (0xff & * U) - 128;
cv = (0xff & * V) - 128;

cr = 104635 * cv;
cg = -25690 * cu + -53294 * cv;
cb = 132278 * cu;

B = LIMIT(cb + cy);
G = LIMIT(cg + cy);
R = LIMIT(cr + cy);

/* One pixel, colors now back in R, G, B */


JP

Archived from groups: rec.video.desktop (More info?)

Jan Panteltje wrote:

>>> Forcing the data to R'G'B' and then back to Y'CbCr again
>>> "just because" does not make any sense.
>>
>> Forgot to add this (in case it is not obvous): "...since
>> each conversion from Y'CbCr to R'G'B' or back is a lossy
>> operation, which degrades the image quality."
>
> Because it is a simple mathematical formula, done forward and
> reverse. You can do it for each pixel!

Of course you can, no-one has disputed that. But it is still a lossy
proces: you lose precision in rounding errors (and even more so if
you clamp the bright or dark end to the "legal" values), and there
are at least two ways to do it - a "full range" conversion and a
"legal range" conversion, which can both be subject to gamma
correction and other not-so-trivial processes which may or may not
be done behind my back. Heck, I do not even know if TMPGEnc (or
whatever component it is that is responsible for the intermediate
R'G'B' conversion) is using the correct weighing factors for the
primaries.

That is a bit too much imprecision and uncertain variables to be
left at the discretion of someone else - in this case, the author
of the encoder. If the data is going to end up inside the MPEG-2
stream in Y'CbCr format, I want to process it in Y'CbCr format
_all the way_, without the encoder interfering and converting it
into R'G'B' space in-between for no good reason.

--
znark

Archived from groups: rec.video.desktop (More info?)

The correct link is of cause:
http://ip51cf87c4.direct-adsl.nl/panteltje/subtitles/
JP

Archived from groups: rec.video.desktop (More info?)

On a sunny day (Thu, 20 May 2004 02:34:21 +0300) it happened "Jukka Aho"
<jukka.aho@iki.fi> wrote in <W7Sqc.2175$DA6.754@reader1.news.jippii.net>:

>Jan Panteltje wrote:
>
>> I dunno TempGEnc, but if it is only an encoder, then yes,
>> stay YUV.
>
>Now why are we discussing this if you do not even have a
>clear idea of what kind of a program TMPGEnc is?
>
>That is the whole problem here; TMPGEnc is a very popular,
>affordable MPEG-2 encoder which, despite its main function
>being only MPEG-2 encoding (and every other function being
>secondary to that main purpose), apparently does not allow
>me to stay in the Y'CbCr colorspace but forces an extra
>conversion step to the R'G'B' colorspace when reading in
>the source clips. (This is contrary to what could be
>expected from a stand-alone software MPEG-2 encoder, but
>that seems to be how the thing works.)
>
>TMPGEnc also features a set of rudimentary color correction,
>masking, cropping, resampling and noise reduction filters
>(which can be switched on and off) but these are only
>secondary "value-add" functions. TMPGEnc is by no means
>intended to be used as a video editor - there is no timeline,
>no editing functions (besides a separate, non-frame-accurate
>MPEG cutting tool); it is basically just a stand-alone
>MPEG-1/2 encoder with some elementary video processing
>filters thrown in.
>
>Some of the built-in filters (if you use them, that is)
>undoubtedly require a conversion into the R'G'B' colorspace
>to work their magic. However, if you never intend to use
>them, converting to R'G'B' "just in case" cannot be a very
>good design decision or clear thinking from the programmers.
>
>> However some things are easier to do in RGB.
>
>Yes, but straight encoding of prepared Y'CbCr material to
>MPEG-2 is not one of those things.
>
>> You will always be working in 8 bits space normally, so 1 /
>> 256 or about 0.4 % accuracy. Nobody can see that.
>
>It is not that simple. Color space conversion between R'G'B'
>and Y'CbCr involves clipping and out-of-gamut colors, not
>merely rounding errors. Some codecs insist on full range
>colours, others hard-clip everything to Rec. 601 range,
>some are adjustable, and yet others do some weird weighted
>scaling at the bright and dark ends, usually in an undocumented
>fashion. Codecs can also have subtle - or sometimes not so
>subtle at all! - differences in how they handle the various
>color subsampling formats, or converting from one subsampling
>format to the other.
>
>Unless you code every single component in your toolchain yourself,
>you can never be quite sure how things work out and what kind of
>algorithms were used - you can only make some educated guesses by
>running some carefully constructed test images through the process.
>That is why every unnecessary conversion step - especially those
>which clearly have no reasonable function and involve closed-source
>programs - is just plain _evil_. If I filter and construct my data
>in the Y'CbCr space and my target format is using Y'CbCr (which
>MPEG-2), I do not want and extra R'G'B' step anywhere in-between,
>without gaining anything from it. (I know my toolchain and the
>filters and programs I use, nothing there technically requires
>converting to R'G'B', it is just the encoder which is forcing
>this on me, for no good reason.)
>
>Anyhow, there are other encoders out there, some of which support
>straight Y'CbCr input (as they logically should!) so the problem
>is not _that_ big nor unsolvable. It just hit me as surprising
>that a widely-used, very popular encoder could be flawed in such
>a technically small (but operationally fundamental) way.
>
>> So again if tempgenc is only a mpeg2 coder, why should it go RGB?
>
>That's the question! I have seen a technical explanation about
>the plugin API (through which everything apparently is imported
>for the encoder to process) not supporting anything else than
>RGB24 format. I have not checked if this really is true.
>
>If it _is_ true, then it technically answers the question, but
>not philosophically: why did they choose to design a plugin
>API which only supports importing files in R'G'B' format -
>for an MPEG encoder!
>
>> Better ask those guys!
>
>"Those guys" are Japanese. It took them a long, long time to
>produce an English version of the program - I remember people
>using unofficially translated versions for a good while before
>they finally officially got someone to translate it to English.
>Getting in direct touch with any of the tech guys (those
>in-the-know) might prove somewhat more challenging than it
>usually is with most other software products... (maybe I
>should start practicing my Japanese!)
>
Hey hey, I have tempgenc (demo) on my machine, but the moment I had
to buy some upgrade to mpeg2 I think it was....WHY
I have Linux and mjpeg tools.
As for RGB to YUV I have shown the math here, there is no magic, you
must be confusing with some other process.
As to black level, there are 2 systems it seems.
In Europe there is no setup (in PAL), while the US uses a setup of a few
percent.
This can have the effect that for example mediaplayer displays too dark,
things made with no setup.
This is in the mpeg2 spec I think:
v = cv * (r - y);
v = (224.0 / 256.0) * v + 128.5; /* 16..240 */


Those modification to RGB->YUV cause confusion, this is not the same
conversion one would do to make a saturation control.
All of this shows clearly one thing:
Go open source, go Linux, get the best, see how it is done, correspond
with the program writer (get your feature implemented, I implemented
quite a few requests), and in English likely.
Big projects like Linux transcode have perhaps almost a hundred people
contributing to it.
While if you buy a windows program you are at the mercy of the company,
one that usually does not release all features (so they can sell an upgrade),
try putting a request with MS!
Or get a bug fixed even, you'd have to wait for the next windows, and
pay for it.

JP

Archived from groups: rec.video.desktop (More info?)

On a sunny day (Thu, 20 May 2004 22:22:09 +0300) it happened "Jukka Aho"
<jukka.aho@iki.fi> wrote in <tx7rc.2355$Jg2.1643@reader1.news.jippii.net>:

>Jan Panteltje wrote:
>
>> Hey hey, I have tempgenc (demo) on my machine,
>
>The name of the program is TMPGEnc, not tempgenc. (TMPGEnc
>is short for "Tsunami MPEG Encoder", but they now use it
>the short form as the official name.)
>
>> As for RGB to YUV I have shown the math here, there is no
>> magic, you must be confusing with some other process.
>
>No matter how many times you keep repeating this, there are
>still great differences in how the commonly used video codecs
>handle these things. Some clamp everything to "legal" values,
>some do not clamp, some have problems with downsampling or
>upsampling the color channels when converting from one format
>to the other, some may have some other anomalies and glitches
>(such as one or half-pixel misplacement in the positioning of
>the color samples.) There are even undocumented and undecided
>things, such as what to do with odd-numbered frame sizes.
>Y'CbCr <-> R'G'B' conversions (or even conversions from one
>Y'CbCr sampling format to the other) are not as straightforward
>in "real life" implementations as you seem to think, and that
>is why every such conversion in the toolchain should be checked
>and double-checked with test images (and, if possible, avoid
>any unnecessary conversions.)
>
>For some real-life examples on this subject and the related
>practical problems, see
>
> <http://www.adamwilt.com/DV-FAQ-editing.html#CodecProblems>
> <http://mywebpages.comcast.net/trbarry/downloads.htm>
> (the ReInterpolate411 filter)
> <http://www.virtualdub.org/> (the "YV12 is b0rked" newsitem)
> <http://www.adamwilt.com/pix-AppleDV.html#Background>
>
>> As to black level, there are 2 systems it seems.
>> In Europe there is no setup (in PAL), while the US
>> uses a setup of a few percent. This can have the
>> effect that for example mediaplayer displays too
>> dark, things made with no setup.
>
>Repeat after me: "setup does not exist in the world of ITU-R
>BT.601 conforming digital video". Whether there is setup or
>not is only a concern of analog playback (also when encoding
>analog signals out of digital video) - but it is a playback-
>time problem in the domain of DACs and analog signals, not a
>digital Y'CbCr data format problem.
>
>But yes, it is yet another source of confusion, and some
>manufacturers have misinterpreted it as well, and some even
>offer an option of _adding setup_ to the digital data stored
>on the tape, which is, of course, wrong:
>
><http://www.adamwilt.com/DV-FAQ-tech.html#Setup>
>
>> This is in the mpeg2 spec I think:
>> v = cv * (r - y);
>> v = (224.0 / 256.0) * v + 128.5; /* 16..240 */
>
>Poynton's Color FAQ is the source to look for the correct formulae:
><http://www.poynton.com/notes/colour_and_gamma/ColorFAQ.html#RTFToC30>
><http://www.poynton.com/notes/colour_and_gamma/ColorFAQ.html#RTFToC31>.
>
>However, it is somehow telling that people who actually implement
>these things (in various tools and such) usually do not look them
>up from reliable sources, or argue with each other about the correct
>way to do it:
>
><http://www.fourcc.org/fccyvrgb.php>
>
>Also note that HDTV formats use different color primaries for the
>Y'CbCr data (as defined in ITU-R BT.709, instead of ITU-R BT.601),
>so it is only going to get worse!
>
>> Go open source, go Linux, get the best, see how it is done,
>> correspond with the program writer (get your feature implemented,
>> I implemented quite a few requests), and in English likely.
>
>I do not have a philosophical agenda with the way how code should
>or should not be written and released; I just use whatever works
>the best for me and what I can afford - often combining commercial,
>open-sourced and freeware tools in my toolchain. This was just one
>of those cases where I could not initially believe that the program
>in question (TMPGEnc is very popular on Windows, which makes it
>even more unbelievable) would not support direct Y'CbCr input.
>
OK, thank you for the lots of links.
It will take some time to check them all out, but of cause it has my
interest.
I agree with most of what you say, yes the different standards I already
mentioned.
I 'grew up' with analog PAL in broadcasting and in that time at least
0.3R + 0.59G + 0.11B we were learned in the school benches.
I took some things from the Berkely mpeg2 codec, its C source is on the
net, some stuff from the webcam soft that was released under GPL
from VLSI Vision Ltd. I think it was, some from what I knew, some from other
places.
Subtitler for example was planned for only adding subs to a mpv es stream,
then I was aksed if I could add a frame counter in screen, then it sort
of fell all together as using linked lists i nC allowed me to work with
any object, added some stuff that seemed useful..
As for 'philosophy' well, this is why open source is not philosophy,
but a very real thing, you ask: 'what and why does tmpgenc (temp from
temporary hehe) do this RGB thing'.
In an open source application you just look at the source, if needed
make your own version! You see this in windows with for example virtualdub,
in Linux many things I wrote have been 'split', 'ported', parts used in
other programs, etc.. and I do the same.
Fine if you release source (GPL).
It benefits everybody that way, I am always happy if some code I wrote is
of use to someone.
But it must also be understood that I write it for myself, for things
I need.
This must be very clear to potential Linux users, unless someone bothers to
write it (say needs it), it won't be there.
Apart from some commercial ventures of cause.
Anyways, lots of interesting stuff, and maybe those links will give me ideas
to improve things.
Appreciated.
Regards
Jan