Sign in with
Sign up | Sign in
Your question

"Raw" file issues?

Last response: in Digital Camera
Share
Anonymous
May 26, 2005 4:31:15 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

http://www.luminous-landscape.com/essays/raw-flaw.shtml

More about : raw file issues

Anonymous
May 26, 2005 4:31:16 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

RichA trolls:

> http://www.luminous-landscape.com/essays/raw-flaw.shtml

http://en.wikipedia.org/wiki/Internet_troll

Some day I will expand on this entry, by noting that "trolling"
behaviours are hardly limited to the Internet -- computer networks did
not magically allow people to suddenly make straw-man arguments.
Consider the classic example of a politician -- our polite name for
those who fabricate false arguments/statements as a professional
activity -- or even the media itself who regularly troll their audience
with prurient, irrelevant, or just plain false information.
Anonymous
May 26, 2005 5:10:49 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

Alan Browne wrote:

> eawckyegcy@yahoo.com wrote:
>
> > activity -- or even the media itself who regularly troll their audience
> > with prurient, irrelevant, or just plain false information.
>
> Don't be an ass. Given the indications that companies like Nikon would
> like nothing better than to sell more s/w as aprt of their camera
> solution (other OEM's too), everyone has everything to gain by a
> standard for RAW images.

It is not in Nikon's interest to tell everyone their innovations in
(say) automatic white balance, or in Canon's interest to spill their
beans re: (say) fancy device physics for optimal bias estimation in
long exposure images.

Reichman, et al, say they don't want "trade secrets" revealed, but they
are woefully clueless in that their demands for documentation of these
files is exactly that.

If you don't like Nikon's or Canon's policies about any of this, you
are free to purchase the products of other companies. Right? Or is
someone forcing you to purchase Nikon's software?

Really, what exactly is the problem here? If anything, the very
existance of Dave Coffin's "dcraw" makes the ranting Reichmann, et al,
look very kookish, and rendering the entire "OpenRaw" issue moot: RAW
file formats are as open as can be, _despite_ the best efforts of
Nikon.
Related resources
Anonymous
May 26, 2005 5:14:34 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

Alan Browne wrote:

> Open RAW is best for everyone in the long term.

48V twisted pair POTS is a "standard" invented at the dawn of the
telephone era. There are many engineers who wish we could rip the
entire mess out of the ground, off the poles and start over again.
Offer praises to Allah or whoever that cell phones, VOIP and the rest
of it are doing the job indirectly.

Be very careful what you wish for.
Anonymous
May 26, 2005 5:47:24 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

Alan Browne wrote:

>> It is not in Nikon's interest to tell everyone their innovations in
>> (say) automatic white balance, or in Canon's interest to spill their
>> beans re: (say) fancy device physics for optimal bias estimation in
>> long exposure images.
>
> No. They give away all the data to "qualified" software developers.

That is a matter for Nikon to decide, not OpenRAW.

> They are not protecting anything to a degree where it would remain a
> secret for more that a week or so. That it can be cracked easilly by
> writers of sw such as dcraw are testimony to the fact that it is pointless.

If it can be "cracked" easily, then OpenRAW has no point.

> > If you don't like Nikon's or Canon's policies about any of this, you
> > are free to purchase the products of other companies. Right? Or is
> > someone forcing you to purchase Nikon's software?
>
> Yes, Nikon. For a person with 000's of dollars in Nikon glass, being
> forced to purchase more beyond a camera is simple wallet gouging.

Nikon is forcing you? Really? Well, if anyone forced me to do
something I didn't want to do I would report the incident as the crime
it is.

> > Really, what exactly is the problem here? If anything, the very
> > existance of Dave Coffin's "dcraw" makes the ranting Reichmann, et al,
> > look very kookish, and rendering the entire "OpenRaw" issue moot: RAW
> > file formats are as open as can be, _despite_ the best efforts of
> > Nikon.
>
> The point is to stop these idiotic and pointless 'races' in the future.

Again, who is forcing you to participate in these "races"?
Anonymous
May 26, 2005 6:06:47 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

Alan Browne wrote:

>> 48V twisted pair POTS is a "standard" invented at the dawn of the
>> telephone era. There are many engineers who wish we could rip the
>> entire mess out of the ground, off the poles and start over again.
>> Offer praises to Allah or whoever that cell phones, VOIP and the rest
>> of it are doing the job indirectly.
>
> Backward compatibilty is not compromised by forward progress where data
> formats are concerned.

Then you must agree that OpenRAW is without a legitimate function.
What's good for the goose, etc.

> Or, if you prefer, open RAW does not mean 'cast in concrete'. Each OEM
> can do as he likes as long as the format is readable by all, and special
> data sections are clearly documented.

As I said, this will in many instances necessarily reveal trade secrets
or other facets of the technology that the manufacturers would likely
be unwilling to disclose. And if part of the OpenRAW is the signing of
an NDA, then can we honestly call it "open"?

> Regarding the olde telephone analog standard it has served extremely
> well for a very long time. Real engineers are too practical to "wish"
> for solutions that don't make economic sense.

If some djinn were to remove the need to work against the crazy POTS
nonsense, many communications engineers would be profoundly thankful.
Indeed, they must be: look at where all the innovation is occuring
today re: telephony. I can cite similar software and hardware
examples. _NO ONE_ likes to deal with legacy systems.

> This backward compatibilty 'tax' is far cheaper than "ripping the
> entire mess out of the ground..."

You are forgetting or ignoring the cost of innovations that are simply
unimplementable within the legacy framework. It is for this and other
reasons that the sensible person does not want camera manufacturers
constrained by some Adobe or OpenRAW or otherwise committee drafted
multi-volume 2356 page standard written in dense legalese.
Anonymous
May 26, 2005 6:18:13 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

Alan Browne wrote:

>> If it can be "cracked" easily, then OpenRAW has no point.
>
> It's the other way around logic-boy.

Ok, then, "OpenRAW has no point if the file formats can be cracked
easily".

You admit the file formats can be cracked easily, therefore ...

>>>>If you don't like Nikon's or Canon's policies about any of this, you
>>>>are free to purchase the products of other companies. Right? Or is
>>>>someone forcing you to purchase Nikon's software?
>>>
>>>Yes, Nikon. For a person with 000's of dollars in Nikon glass, being
>>>forced to purchase more beyond a camera is simple wallet gouging.
>>
>>
>> Nikon is forcing you? Really? Well, if anyone forced me to do
>> something I didn't want to do I would report the incident as the crime
>> it is.
>
> Oh sheesh. It's not like you decide one day "I hate those Mazda
> bastards" and change car brands. Replacing a lens collection is not
> trivial. Nikon have a lot of loyal customers who are now having part of
> their own images encrypted and requiring Nikon authorized s/w to read it.

Nikon can also change their lens mount. What are you going to do, sue
them? After OpenRAW has passed their legislation mandating The File
Format, is the next step to mandate interoperable optics?

> >>The point is to stop these idiotic and pointless 'races' in the future.
> >
> > Again, who is forcing you to participate in these "races"?
>
> Nikon. By encrypting part of MY image without my consent.

You consented to it by pressing the shutter button.
Anonymous
May 26, 2005 6:42:01 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

Alan Browne wrote:

>> Ok, then, "OpenRAW has no point if the file formats can be cracked
>> easily".
>>
>> You admit the file formats can be cracked easily, therefore ...
>
> Since it is all but impossible to encrypt the data in the first place,
> there is no need to do so.

Again, that is something for the manufacturers to decide. (Personally,
I agree with you and likely the guy who wrote the firmware for the D70,
D2x, etc is also in violent agreement. It's the dingbats in marketing,
etc, who are ignorant.)

> Or I suppose the OEM's could begin keying
> the encryption on a case by case basis. Yeah, uh huh.

As you know, it wouldn't matter. www.google.com: softice debugger (and
so forth).

>> Nikon can also change their lens mount. What are you going to do, sue
>> them? After OpenRAW has passed their legislation mandating The File
>> Format, is the next step to mandate interoperable optics?
>
> A good point that misses the point. Nikon have made big marketing hooey
> over backward comaptible lens mounts. So in revenge they're making part
> of their files unreadable? Yeah, okay. People will buy that. Sure.

Nikon is free to make fools of themselves in the market. Who are we to
complain?

>> You consented to it by pressing the shutter button.
>
> Nobody considers this when they maintain Nikon lens collections or
> purcahse a Nikon camera. Or other OEM's for that matter.

We all make mistakes. Sometimes big ones.

> End of thread for me.

Hasta luego!
Anonymous
May 26, 2005 7:20:56 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

Alan Browne wrote:

>>>Backward compatibilty is not compromised by forward progress where data
>>>formats are concerned.
>>
>> Then you must agree that OpenRAW is without a legitimate function.
>> What's good for the goose, etc.
>
> I don't follow that one.

Hey, you made the claim. If your statement is true, then the camera
makers are free to invent whatever new file formats they like ("forward
progress") since "backward compatibility is not compromised". And with
such freedom, what is OpenRAW's purpose?

> What makes sense is that what gets recorded
> onto the memory card is the property of the photographer or employer,
> not the camera OEM.

Sounds like you need an IP attorney, not an OpenRAW standard.

>> As I said, this will in many instances necessarily reveal trade secrets
>> or other facets of the technology that the manufacturers would likely
>> be unwilling to disclose. And if part of the OpenRAW is the signing of
>> an NDA, then can we honestly call it "open"?
>
> Since these trade secrets have a half life in the wild of about 3 days,
> they are no more useful to the OEM's than encryption has been for the
> DVD producers. NDA's don't protect anything of this nature.

Yes, yes, the manufacturers can make idiots of themselves. Why are you
so concerned about this?

> "Trade Secret"? Why hasn't the dcraw author been sued into
> homelessness?

He hasn't published anything particularly sensitive yet? But even if
he did, trade secrets, in the absence of a contract, have basically no
legal protections at all.

> If he can do it, the folks at competing OEM's can do it
> just as quick.

Reverse engineering is an honoured tradition. What better compliment
can you offer the manufacturer?

>> If some djinn were to remove the need to work against the crazy POTS
>> nonsense, many communications engineers would be profoundly thankful.
>
> But that's not engineering.

Of course it is. There are many situations, particularly in software,
where one invents brand new stuff all the time. No legacy constraints.
It is a most refreshing experience.

> The same could be said about many
> infrastructures from roadways to airways.

Your point?

> If we were to design "World
> 2.0" do you think we would have LA, Mexico and Cairo (to name very few).

I honestly couldn't say. And really, is this relevant?

>> Indeed, they must be: look at where all the innovation is occuring
>> today re: telephony. I can cite similar software and hardware
>> examples. _NO ONE_ likes to deal with legacy systems.
>
> If it is economically sensible to do so, it is done. That is
> engineering in large part. "Like" has little to with it.

"Like" has everything to do with it, since if nobody likes it, it won't
get done. More importantly, though, if physical reality says "no", it
doesn't matter what you think.

>>>This backward compatibilty 'tax' is far cheaper than "ripping the
>>>entire mess out of the ground..."
>>
>>
>> You are forgetting or ignoring the cost of innovations that are simply
>> unimplementable within the legacy framework. It is for this and other
>> reasons that the sensible person does not want camera manufacturers
>> constrained by some Adobe or OpenRAW or otherwise committee drafted
>> multi-volume 2356 page standard written in dense legalese.
>
> See above, re: World 2.0.

Ok, then, send Canon a check to cover their costs of abiding by the
Official OpenRAW Standard, or to make up for lost revenue to products
or services which they would like to produce, but which can not fit
into this standard.

> If that's so, why are they all committe members, participants and
> signatories to various other standards?

Oh, I could rant about this at some length. But why should I entertain
your distractions?

> Feel free to have the last word ... this is it for me.

Thanks!
Anonymous
May 26, 2005 7:29:54 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

eawckyegcy@yahoo.com wrote:

> activity -- or even the media itself who regularly troll their audience
> with prurient, irrelevant, or just plain false information.

Don't be an ass. Given the indications that companies like Nikon would
like nothing better than to sell more s/w as aprt of their camera
solution (other OEM's too), everyone has everything to gain by a
standard for RAW images.


--
-- r.p.e.35mm user resource: http://www.aliasimages.com/rpe35mmur.htm
-- r.p.d.slr-systems: http://www.aliasimages.com/rpdslrsysur.htm
-- [SI] gallery & rulz: http://www.pbase.com/shootin
-- e-meil: Remove FreeLunch.
Anonymous
May 26, 2005 8:20:09 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

eawckyegcy@yahoo.com wrote:
[snip]
> Ok, then, send Canon a check to cover their costs of abiding by the
> Official OpenRAW Standard, or to make up for lost revenue to products
> or services which they would like to produce, but which can not fit
> into this standard.
[snip]

The cost of abiding by a common Raw format would probably be small.
There would probably be a start-up cost - "climbing the learning
curve". (If I were a manager in a camera-making company, I would prefer
to manage the second camera in the company to use it, not the first!)

After that, it would probably be cheaper. There might be cost-savings
such as common bits of firmware that could be bought, re-used, etc. And
savings in learning about the format. There has been a lot of "wheel
re-inventing" up to now, with must surely add to cost, and perhaps
reduce quality.

No camera launched in the last year and a half would have needed a
change to a well-engineered common Raw format. The vast majority of
changes, such as more pixels, greater bit depth, etc, don't need
changes. So there may not be any such products and services that they
couldn't produce.

The one restriction that camera manufacturers would face is that they
could not then use encryption to coerce their customers to buy their
software.

--
Barry Pearson
http://www.barry.pearson.name/photography/
http://www.birdsandanimals.info/
Anonymous
May 26, 2005 8:26:40 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

eawckyegcy@yahoo.com wrote:

>
> It is not in Nikon's interest to tell everyone their innovations in
> (say) automatic white balance, or in Canon's interest to spill their
> beans re: (say) fancy device physics for optimal bias estimation in
> long exposure images.

No. They give away all the data to "qualified" software developers.
They are not protecting anything to a degree where it would remain a
secret for more that a week or so. That it can be cracked easilly by
writers of sw such as dcraw are testimony to the fact that it is pointless.

> If you don't like Nikon's or Canon's policies about any of this, you
> are free to purchase the products of other companies. Right? Or is
> someone forcing you to purchase Nikon's software?

Yes, Nikon. For a person with 000's of dollars in Nikon glass, being
forced to purchase more beyond a camera is simple wallet gouging.

> Really, what exactly is the problem here? If anything, the very
> existance of Dave Coffin's "dcraw" makes the ranting Reichmann, et al,
> look very kookish, and rendering the entire "OpenRaw" issue moot: RAW
> file formats are as open as can be, _despite_ the best efforts of
> Nikon.

The point is to stop these idiotic and pointless 'races' in the future.
When you buy a film camera, it works with the specified film. No
special to Nikon processes are required. Nikon are now requiring
special to type processes for their digital cameras.

--
-- r.p.e.35mm user resource: http://www.aliasimages.com/rpe35mmur.htm
-- r.p.d.slr-systems: http://www.aliasimages.com/rpdslrsysur.htm
-- [SI] gallery & rulz: http://www.pbase.com/shootin
-- e-meil: Remove FreeLunch.
Anonymous
May 26, 2005 8:36:04 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

eawckyegcy@yahoo.com wrote:

> Alan Browne wrote:
>
>
>>Open RAW is best for everyone in the long term.
>
>
> 48V twisted pair POTS is a "standard" invented at the dawn of the
> telephone era. There are many engineers who wish we could rip the
> entire mess out of the ground, off the poles and start over again.
> Offer praises to Allah or whoever that cell phones, VOIP and the rest
> of it are doing the job indirectly.

Backward compatibilty is not compromised by forward progress where data
formats are concerned. Formats can get ever richer without s/w losing
the ability to read old data correctly. That is if s/w and formats are
designed and maintained correctly.

Or, if you prefer, open RAW does not mean 'cast in concrete'. Each OEM
can do as he likes as long as the format is readable by all, and special
data sections are clearly documented.

Regarding the olde telephone analog standard it has served extremely
well for a very long time. Real engineers are too practical to "wish"
for solutions that don't make economic sense. They adapt instead to
growth without leaving anything of value behind. This backward
compatibilty 'tax' is far cheaper than "ripping the entire mess out of
the ground..."

--
-- r.p.e.35mm user resource: http://www.aliasimages.com/rpe35mmur.htm
-- r.p.d.slr-systems: http://www.aliasimages.com/rpdslrsysur.htm
-- [SI] gallery & rulz: http://www.pbase.com/shootin
-- e-meil: Remove FreeLunch.
Anonymous
May 26, 2005 8:51:35 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

eawckyegcy@yahoo.com wrote:
[snip]
> As I said, this will in many instances necessarily reveal trade secrets
> or other facets of the technology that the manufacturers would likely
> be unwilling to disclose. And if part of the OpenRAW is the signing of
> an NDA, then can we honestly call it "open"?

Publicising current formats may reveal some things that manufacturers
would not like revealed. Although that may be just the design quality
of their Raw file formats! But interface specifications don't normally
reveal much about the inside of the box. I would expect the formats to
identify the sensor configuration, but the manufacturers are typically
willing, sometimes eager, to tell the world about that anyway.

Use of a common Raw format would reveal even less. At the moment, the
DNG format caters for Bayer sensors, Fujifilm SR sensors, Sony 4-colour
sensors, and Foveon sensors. (Possibly lots more configurations could
also be handled). If you can handle all of those with a single
specification, then using that specification isn't revealing much about
your technology. Formats designed specifically for your technology are
more likely to be revealing, if only accidentally.

[snip]
> You are forgetting or ignoring the cost of innovations that are simply
> unimplementable within the legacy framework. It is for this and other
> reasons that the sensible person does not want camera manufacturers
> constrained by some Adobe or OpenRAW or otherwise committee drafted
> multi-volume 2356 page standard written in dense legalese.

What innovations are those? No new cameras in the last year and a half
would have needed a change to a common Raw format. The vast majority of
innovations are not visible in details of the Raw file format.

The DNG specification is 50 pages long, and a lot of that is white
space. It builds on TIFF/EP (the draft was 65 pages). But the camera
manufacturers used that anyway! Typical proprietary Raw formats are
already based on TIFF/EP. (And TIFF/EP refers to others). At the
moment, internally the manufacturers must use these external
specifications plus their own internal usage specifications. DNG, or
other common Raw format, would replace some internal specification.

Any photographer concerned with the future health of top-end digital
photography ought to want camera manufacturers to use a common Raw
format. And DNG is suitable for that purpose.

--
Barry Pearson
http://www.barry.pearson.name/photography/
http://www.birdsandanimals.info/
Anonymous
May 26, 2005 8:56:01 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

eawckyegcy@yahoo.com wrote:

>
> If it can be "cracked" easily, then OpenRAW has no point.

It's the other way around logic-boy.

>
>
>>>If you don't like Nikon's or Canon's policies about any of this, you
>>>are free to purchase the products of other companies. Right? Or is
>>>someone forcing you to purchase Nikon's software?
>>
>>Yes, Nikon. For a person with 000's of dollars in Nikon glass, being
>>forced to purchase more beyond a camera is simple wallet gouging.
>
>
> Nikon is forcing you? Really? Well, if anyone forced me to do
> something I didn't want to do I would report the incident as the crime
> it is.

Oh sheesh. It's not like you decide one day "I hate those Mazda
bastards" and change car brands. Replacing a lens collection is not
trivial. Nikon have a lot of loyal customers who are now having part of
their own images encrypted and requiring Nikon authorized s/w to read it.

>>The point is to stop these idiotic and pointless 'races' in the future.
>
>
> Again, who is forcing you to participate in these "races"?

Nikon. By encrypting part of MY image without my consent.

Cheers,
Alan.


--
-- r.p.e.35mm user resource: http://www.aliasimages.com/rpe35mmur.htm
-- r.p.d.slr-systems: http://www.aliasimages.com/rpdslrsysur.htm
-- [SI] gallery & rulz: http://www.pbase.com/shootin
-- e-meil: Remove FreeLunch.
Anonymous
May 26, 2005 9:02:07 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

eawckyegcy@yahoo.com wrote:
> Alan Browne wrote:
>
> > Open RAW is best for everyone in the long term.
>
> 48V twisted pair POTS is a "standard" invented at the dawn of the
> telephone era. There are many engineers who wish we could rip the
> entire mess out of the ground, off the poles and start over again.
> Offer praises to Allah or whoever that cell phones, VOIP and the rest
> of it are doing the job indirectly.

Perhaps, without that standard, we would not be advanced enough to make
that wish! We would still be wasting too much time trying to get basic
communications to work.

We sort out one layer of the communications interworking. That enables
us to build lots of services in higher layers. That creates a desire
for better lower layers. The trick is to have an architecture, or
structural decomposition, that enable to make progress in one area
without having to tear up all other areas. And that needs clean
interfaces between the various areas.

If we DO decide to rip all that stuff out, I would hope we could do so
without (say) having to re-invent TCP/IP to run over its successor.

> Be very careful what you wish for.

I would think that anyone who has spent many years or decades helping
to design complex multi-vendor systems knows what to wish for here! We
need a common Raw format. And in the meantime we need sufficient
publication of our current formats to build stop-gap measures such as
access to proprietary Raw formats, and conversion to a common Raw
format, etc.

--
Barry Pearson
http://www.barry.pearson.name/photography/
http://www.birdsandanimals.info/
Anonymous
May 26, 2005 9:24:15 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

Barry Pearson wrote:

> Publicising current formats may reveal some things that manufacturers
> would not like revealed. Although that may be just the design quality
> of their Raw file formats!

I can't comment on Nikon stuff, since I'm a Canon sort of person.

Canon 10D: a bizarre CIFF, for which they published the specifications
for the "superstructure" of this file. It's a kind of weird TIFF; you
can, in fact, TIFFize a CIFF. There are two images in this file: the
JPEG and an strangely encoded raw sensor dump. That Dave Coffin
managed to figure it out speaks for his abilities as a Reverse
Engineer.

Canon 1DMkII: it's a straight TIFF, with EXIF markup. Three images
embedded: a flat RGB thumbnail, a 1536 pixel wide JPEG (used for
in-camera editing), and the lossless JPEG encoded sensor dump.

The main mysteries for both are not the bulk format, but the
undocumented tags, specifically the "EXIF MakerNote" as well as the
various non-standard TIFF tags.

> But interface specifications don't normally
> reveal much about the inside of the box. I would expect the formats to
> identify the sensor configuration, but the manufacturers are typically
> willing, sometimes eager, to tell the world about that anyway.

I have reverse engineered a fair amount of the 10D's undocumented tags
(and some of the 1DMkII's -- too busy). There is one that does indeed
lay out of the geometry of the sensor (how large, where the "image"
starts, and so forth). There are other tags that detail the active AF
sensor bitmap, various light meter readings, exposure computations and
so forth. Just a few days ago, I found (by accident) the default
luminance curve in the 1DMkII CR2 file.

Probably the most interesting aspect is one that is relevant here: the
sensor has a 64 column bitmap "black" pixels on the left side of the
image. The usual assumption is that these are used for bias level
stuff. However, there is unusual stuff occuring here when the
exposures grow to 30 seconds or more ... the 64 columns splits into
distinct 16 column set and a 48 column set, and these columns are
(apparently) used to produce a much better bias estimate than would be
obtained from simple averaging.

This is precisely the sort of detail that Canon would almost certainly
_not_ want to be generally known (it took me a while to figure it out),
not just that it exists, but that someone might be able to figure out
the device physics and go from there. If this is such a thing, though,
there are almost certainly others awaiting.

Or maybe everyone knows this stuff because it is a Canon patented
process.

> Use of a common Raw format would reveal even less.

Suppose the above 16/48 column stuff was a Canon "trade secret" they
would prefer to keep. How could a standard format encompass such
sensor behaviour without revealing the secret? Or suppose that
everyone knows this -- it is a Canon patent. Unless Canon licenses
this patent for zero cost (which we can assume will be a
zero-probability event), no one can use it even if it was fully
documented to the last bit.

> Formats designed specifically for your technology are
> more likely to be revealing, if only accidentally.

I suggest this is more common than people think. The "MakerNote"
information is tightly coupled to the hardware, at least it is for
Canon equipment.
Anonymous
May 26, 2005 9:28:42 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

eawckyegcy@yahoo.com wrote:
> Alan Browne wrote:
>
>
>>>If it can be "cracked" easily, then OpenRAW has no point.
>>
>>It's the other way around logic-boy.
>
>
> Ok, then, "OpenRAW has no point if the file formats can be cracked
> easily".
>
> You admit the file formats can be cracked easily, therefore ...

Since it is all but impossible to encrypt the data in the first place,
there is no need to do so. Or I suppose the OEM's could begin keying
the encryption on a case by case basis. Yeah, uh huh.

>>
>>Oh sheesh. It's not like you decide one day "I hate those Mazda
>>bastards" and change car brands. Replacing a lens collection is not
>>trivial. Nikon have a lot of loyal customers who are now having part of
>>their own images encrypted and requiring Nikon authorized s/w to read it.
>
>
> Nikon can also change their lens mount. What are you going to do, sue
> them? After OpenRAW has passed their legislation mandating The File
> Format, is the next step to mandate interoperable optics?

A good point that misses the point. Nikon have made big marketing hooey
over backward comaptible lens mounts. So in revenge they're making part
of their files unreadable? Yeah, okay. People will buy that. Sure.


>>>>The point is to stop these idiotic and pointless 'races' in the future.
>>>
>>>Again, who is forcing you to participate in these "races"?
>>
>>Nikon. By encrypting part of MY image without my consent.
>
>
> You consented to it by pressing the shutter button.

Nobody considers this when they maintain Nikon lens collections or
purcahse a Nikon camera. Or other OEM's for that matter.

End of thread for me.

Cheers,
Alan


--
-- r.p.e.35mm user resource: http://www.aliasimages.com/rpe35mmur.htm
-- r.p.d.slr-systems: http://www.aliasimages.com/rpdslrsysur.htm
-- [SI] gallery & rulz: http://www.pbase.com/shootin
-- e-meil: Remove FreeLunch.
Anonymous
May 26, 2005 9:37:13 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

eawckyegcy@yahoo.com wrote:

> Alan Browne wrote:
>
>
>>>48V twisted pair POTS is a "standard" invented at the dawn of the
>>>telephone era. There are many engineers who wish we could rip the
>>>entire mess out of the ground, off the poles and start over again.
>>>Offer praises to Allah or whoever that cell phones, VOIP and the rest
>>>of it are doing the job indirectly.
>>
>>Backward compatibilty is not compromised by forward progress where data
>>formats are concerned.
>
>
> Then you must agree that OpenRAW is without a legitimate function.
> What's good for the goose, etc.

I don't follow that one. What makes sense is that what gets recorded
onto the memory card is the property of the photographer or employer,
not the camera OEM.

>
>>Or, if you prefer, open RAW does not mean 'cast in concrete'. Each OEM
>>can do as he likes as long as the format is readable by all, and special
>>data sections are clearly documented.
>
>
> As I said, this will in many instances necessarily reveal trade secrets
> or other facets of the technology that the manufacturers would likely
> be unwilling to disclose. And if part of the OpenRAW is the signing of
> an NDA, then can we honestly call it "open"?

Since these trade secrets have a half life in the wild of about 3 days,
they are no more useful to the OEM's than encryption has been for the
DVD producers. NDA's don't protect anything of this nature.

"Trade Secret"? Why hasn't the dcraw author been sued into
homelessness? If he can do it, the folks at competing OEM's can do it
just as quick.


>
>
>>Regarding the olde telephone analog standard it has served extremely
>>well for a very long time. Real engineers are too practical to "wish"
>>for solutions that don't make economic sense.
>
>
> If some djinn were to remove the need to work against the crazy POTS
> nonsense, many communications engineers would be profoundly thankful.

But that's not engineering. The same could be said about many
infrastructures from roadways to airways. If we were to design "World
2.0" do you think we would have LA, Mexico and Cairo (to name very few).

> Indeed, they must be: look at where all the innovation is occuring
> today re: telephony. I can cite similar software and hardware
> examples. _NO ONE_ likes to deal with legacy systems.

If it is economically sensible to do so, it is done. That is
engineering in large part. "Like" has little to with it.

>
>
>>This backward compatibilty 'tax' is far cheaper than "ripping the
>>entire mess out of the ground..."
>
>
> You are forgetting or ignoring the cost of innovations that are simply
> unimplementable within the legacy framework. It is for this and other
> reasons that the sensible person does not want camera manufacturers
> constrained by some Adobe or OpenRAW or otherwise committee drafted
> multi-volume 2356 page standard written in dense legalese.

See above, re: World 2.0.

If that's so, why are they all committe members, participants and
signatories to various other standards?

Feel free to have the last word ... this is it for me.

Cheers,
Alan


--
-- r.p.e.35mm user resource: http://www.aliasimages.com/rpe35mmur.htm
-- r.p.d.slr-systems: http://www.aliasimages.com/rpdslrsysur.htm
-- [SI] gallery & rulz: http://www.pbase.com/shootin
-- e-meil: Remove FreeLunch.
Anonymous
May 26, 2005 9:47:28 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

Alan Browne wrote:


> infrastructures from roadways to airways. If we were to design "World
> 2.0" do you think we would have LA, Mexico and Cairo (to name very few).

oops. Meant "Mexico City".


--
-- r.p.e.35mm user resource: http://www.aliasimages.com/rpe35mmur.htm
-- r.p.d.slr-systems: http://www.aliasimages.com/rpdslrsysur.htm
-- [SI] gallery & rulz: http://www.pbase.com/shootin
-- e-meil: Remove FreeLunch.
Anonymous
May 26, 2005 10:29:47 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

Owamanga wrote:

> On Thu, 26 May 2005 12:31:15 -0400, RichA <none@none.com> wrote:
>
>
>>http://www.luminous-landscape.com/essays/raw-flaw.shtml
>
>
> Chicken liken twaddle.

Open RAW is best for everyone in the long term. There's no excuse for
any data in the image created by the camera you own to be obfuscated or
encrypted.


--
-- r.p.e.35mm user resource: http://www.aliasimages.com/rpe35mmur.htm
-- r.p.d.slr-systems: http://www.aliasimages.com/rpdslrsysur.htm
-- [SI] gallery & rulz: http://www.pbase.com/shootin
-- e-meil: Remove FreeLunch.
Anonymous
May 26, 2005 10:49:59 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

Owamanga wrote:

> On Thu, 26 May 2005 15:33:51 -0400, Alan Browne
> <alan.browne@freelunchVideotron.ca> wrote:
>
>
>>Owamanga wrote:
>>
>>
>>>http://www.luminous-landscape.com/essays/raw-flaw.shtml
>>>
>>>Chicken liken twaddle.
>>
>>Open RAW is best for everyone in the long term. There's no excuse for
>>any data in the image created by the camera you own to be obfuscated or
>>encrypted.
>
>
> With that I agree, my complaint is with stupidity such as this:
>
> "What happens when your new Quatum Cube based computer no longer can
> read CDs, or DVDs, or its operating system can't deal with something
> as old and arcane as Windows XP or Mac OSX?
>
> Far fetched you say? We'll, how many of you have a stack of 3.5"
> floppies somewhere in your desk drawer, and when was the last time you
> had a floppy disk drive attached to your computer? Still do you say?

Yep. Even test it every 6 mo. or so. I have floppies, but the data has
long since been migrated via Zip and CD-ROM. My PGP private keys are
backed up on floppy as well, but I suppose a CD-ROM transition wouldn't
be a bad idea.

> OK. How about 5.25" floppies? Bet you don't have one of these sitting
> around anymore, except maybe moldering away in the basement
> somewhere."

Trashed.

>
> Now tell me Alan, how does Open RAW suddenly enable a 3.5" floppy disk
> to fit into a DVD drive?

Irrelevant to the RAW issue.

I have backups from early 80's 5.25" floppies ... on CD-ROM. As the
storage technology has moved forward, so has my backup set as we do tend
to have the recent and newest storage devices in our system at the same
time. Sounds like a pain, but as none of the media to data have proven
able to backup for more than about 5 years, the updating of

Now I'm in the unhurried process of moving backups to DVD. After DVD
there will be something else. In between I've used 3.5" as backup and
Zip. So I will, in time, have DVD backups with 1980's COSMAC 1802,
Motorola 6800 and other micro assembler code, FORTRAN (IV, WATFOR, 77),
PASCAL, Modula, Ada and others on them to entertain me in my old age...

For my Maxxum 7D, I've been backing up the camera RAW, and I have little
doubt that it will be decodable in 100 years under "Microtosh WindOSnix
FarFetched Episode Seven Patch 19, for 8192 bit processors on
n-web-processor-distributed threads, Release 12.98.34", should anyone
choose to do so. Won't be me barring miracles.

Hence my big yawn over DNG ... until I discovered the other day in an
actual experiment with real files that DNG takes 26 to 41% less space
than .MRW. So I'm mulling the transition to DNG solely to reduce the
DVD count by 1 for every 3 or 4 DVD's. I haven't committed to the
decision yet.

But prior to that, my DNG 'objection' was: when the OEM's adopt it, I'll
adopt it.

Cheers,
Alan



--
-- r.p.e.35mm user resource: http://www.aliasimages.com/rpe35mmur.htm
-- r.p.d.slr-systems: http://www.aliasimages.com/rpdslrsysur.htm
-- [SI] gallery & rulz: http://www.pbase.com/shootin
-- e-meil: Remove FreeLunch.
Anonymous
May 27, 2005 1:55:50 AM

Archived from groups: rec.photo.digital.slr-systems (More info?)

On 26 May 2005 13:10:49 -0700, "eawckyegcy@yahoo.com"
<eawckyegcy@yahoo.com> wrote:

>Alan Browne wrote:
>
>> eawckyegcy@yahoo.com wrote:
>>
>> > activity -- or even the media itself who regularly troll their audience
>> > with prurient, irrelevant, or just plain false information.
>>
>> Don't be an ass. Given the indications that companies like Nikon would
>> like nothing better than to sell more s/w as aprt of their camera
>> solution (other OEM's too), everyone has everything to gain by a
>> standard for RAW images.
>
>It is not in Nikon's interest to tell everyone their innovations in
>(say) automatic white balance, or in Canon's interest to spill their
>beans re: (say) fancy device physics for optimal bias estimation in
>long exposure images.
>
>Reichman, et al, say they don't want "trade secrets" revealed, but they
>are woefully clueless in that their demands for documentation of these
>files is exactly that.
>
>If you don't like Nikon's or Canon's policies about any of this, you
>are free to purchase the products of other companies. Right? Or is
>someone forcing you to purchase Nikon's software?
>
>Really, what exactly is the problem here? If anything, the very
>existance of Dave Coffin's "dcraw" makes the ranting Reichmann, et al,
>look very kookish, and rendering the entire "OpenRaw" issue moot: RAW
>file formats are as open as can be, _despite_ the best efforts of
>Nikon.

Here we have an example of a prime case of the toadying
"company man" not concerned with customers. It's a disease
that afflicts many people who work in companies over 1000 people,
or those who fantasize that their own work is more valuable than
good customer service.
-Rich
Anonymous
May 27, 2005 2:08:27 AM

Archived from groups: rec.photo.digital.slr-systems (More info?)

I have had to help computer users who changed computers and suddenly
couldn't open their old files any more. It isn't pretty. Those
who pooh-pooh these kinds of problems have probably never had to
deal with them. Or maybe the whole concept of planning for the
long term is alien to some people.

--
Ben Rosengart (212) 741-4400 x215
Sometimes it only makes sense to focus our attention on those
questions that are equal parts trivial and intriguing.
--Josh Micah Marshall
Anonymous
May 27, 2005 2:20:54 AM

Archived from groups: rec.photo.digital.slr-systems (More info?)

In article <d75bdp$ga7$1@inews.gazeta.pl>,
Alan Browne <alan.browne@FreelunchVideotron.ca> wrote:
>eawckyegcy@yahoo.com wrote:

[ ... ]

>> Really, what exactly is the problem here? If anything, the very
>> existance of Dave Coffin's "dcraw" makes the ranting Reichmann, et al,
>> look very kookish, and rendering the entire "OpenRaw" issue moot: RAW
>> file formats are as open as can be, _despite_ the best efforts of
>> Nikon.
>
>The point is to stop these idiotic and pointless 'races' in the future.
> When you buy a film camera, it works with the specified film. No
>special to Nikon processes are required. Nikon are now requiring
>special to type processes for their digital cameras.

When you buy a film camera, you are locked into the physical
format of the film. If you want to shoot on some Royal-X Pan (very high
speed B&W film -- probably now long dead), it was available in 120 and
620 formats, but *not* in 35mm. (For good reason, as the grain would
have been excessive in 35mm.)

Using 120 roll film or 4x5 sheet film in a 35mm camera is not
practical (though there were some strange adaptors in existence,
including one to take a 35mm image from the back of a Nikon F and expand
it to expose a 4x5 sheet film, or a 4x5 Polaroid (usually 3000 ASA).

And to use other films, such as Infrared Ektachrome in your 35mm
SLR, you have to put some rather extreme filtration on it -- another
example of special processing. (Granted, the filters are available form
other vendors, but the basic fact that your film camera will not work
with this film without special processing (the filters) remains.

As for another phase of the same problem, when you buy a given
film, you may be required to buy a certain set of chemicals to process
that film -- much equivalent to requiring special software to process a
RAW format. (Except that with the digital cameras, you are not *forced*
to use the RAW format -- the camera will happily provide you with JPEGs
(the equivalent of carrying the film to the corner drugstore for
processing.)

But you could not process Kodak Ektachrome with Agfa chemicals
or Ansco chemicals. For that matter, you could not process Kodak's
Kodachrome with Kodak's Ektachrome chemicals -- let alone processing
Kodacolor.

Ektachrome even changed processes over time. Sometimes the new
chemical set would work with the older film, and sometimes not. I know
this, because I used to develop my own Ektachrome a *lot* -- from
perhaps around 1962 on through about 1976, when I got married, and was
working for a government organization which prohibited my bringing
cameras on base with me, so my photo work pretty much stopped until I
retired, and then the digital cameras came along.

And there was at least one very high resolution B&W film which
required a special developer (was the maker H&W? I used it -- once.)

Granted -- for most B&W films, you could use a fairly wide range
of chemicals from different vendors to develop it --- including mixing
your own from old formulas -- or from knowing enough about the process
to devise your own. (There was not enough information about the slide
films to allow this freedom. And it is a lot harder to pick apart the
photochemistry of a transparency film than an image format.)

But -- I don't see how the situation with RAW files is any worse
than these limitations from the film days. And in some ways, it is a
*lot* better. If you process a RAW file with the wrong software, the
worst that will happen is that you won't get anything useful. The
original RAW file can be saved until you get the *right* software.

Note that I prefer to do my image processing on unix computers,
which are *not* supported by either Kodak or Cannon (or any of the other
digital SLR makers, AFIK). For this reason, I am quite pleased to have
Dave Coffin's "dcraw" software available.

Yes -- having all the vendors use a common format would be
convenient (other than JPEG, which I detest as a lossy algorithm), but
I'll expect that to happen about the time that all film maker's color
processes are interchangeable. :-)

Enjoy,
DoN.
--
Email: <dnichols@d-and-d.com> | Voice (all times): (703) 938-4564
(too) near Washington D.C. | http://www.d-and-d.com/dnichols/DoN.html
--- Black Holes are where God is dividing by zero ---
Anonymous
May 27, 2005 2:46:21 AM

Archived from groups: rec.photo.digital.slr-systems (More info?)

In article <1117143721.412572.7590@o13g2000cwo.googlegroups.com>,
eawckyegcy@yahoo.com <eawckyegcy@yahoo.com> wrote:
>Alan Browne wrote:
>
>>> Ok, then, "OpenRAW has no point if the file formats can be cracked
>>> easily".
>>>
>>> You admit the file formats can be cracked easily, therefore ...
>>
>> Since it is all but impossible to encrypt the data in the first place,
>> there is no need to do so.
>
>Again, that is something for the manufacturers to decide. (Personally,
>I agree with you and likely the guy who wrote the firmware for the D70,
>D2x, etc is also in violent agreement. It's the dingbats in marketing,
>etc, who are ignorant.)
>
>> Or I suppose the OEM's could begin keying
>> the encryption on a case by case basis. Yeah, uh huh.
>
>As you know, it wouldn't matter. www.google.com: softice debugger (and
>so forth).

If they wanted to be *serious* about the encryption, I don't
think that "softice" could do much about it.

Start with a public-key/private-key algorithm, similar to what
is ussed in PGP.

Embed the public key in the EXIF data, similar to the serial
number and model number (except that it would take significantly more
bytes of data).

Add a private key to get which you have to send to the vendor
both an image from your camera (to get the EXIF data), and a unique
number in the computer (hostID on Sun workstations, something else in
the firmware on a PC or a Mac). In exchange, you will get a key which
will work only with *your* camera on *your* computer. (It should be
possible to have the computer carrying the keys for several cameras, and
to recognize which is needed by examining the EXIF data. You could even
do whole image encryption.

But -- you have to be willing to pay the costs:

1) The file size will grow in the process of encryption.

2) The CPU usage in the camera would *greatly* increase -- cutting
into the frames per second rate in burst mode, and shortening
the battery life.

3) The camera will be useless to anyone else.


There are *some* benefits from this.

1) If you are working in a sensitive field, if the camera (or the
media) are stolen, the images are not compromised.

2) If the disk drives from the computer are stolen, that will
not be sufficient to use the images stored on it. (At least not
the RAW ones.)

3) If the camera is stolen, there will be minimal market for it,
without the key -- and to get another copy of the key from the
manufacturer, you would need to expose who now had the camera,
so stolen cameras would be easier to retrieve. (The
manufacturer would require the consent of the registered owner
to release new keys, so if you are not the registered owner, you
are likely the thief.)

Ideally, however, this sort of encryption would be turned off by
default, and would require an explicit action (similar to installing a
firmware patch) to turn it on. (Thus -- only those who feel that they
could benefit from the encryption would have to pay the costs listed
above.)

Under those circumstances, you could bypass the computer hostid
(or equivalent) part to the key, and supply private key and public key
in the same download -- to be separated in the computer which installs
the firmware in the camera. It could then modify itself (if so desired)
to lock to a single computer.

Enjoy,
DoN.
--
Email: <dnichols@d-and-d.com> | Voice (all times): (703) 938-4564
(too) near Washington D.C. | http://www.d-and-d.com/dnichols/DoN.html
--- Black Holes are where God is dividing by zero ---
Anonymous
May 27, 2005 4:29:43 AM

Archived from groups: rec.photo.digital.slr-systems (More info?)

eawckyegcy@yahoo.com wrote:
[snip]
> I can't comment on Nikon stuff, since I'm a Canon sort of person.
>
> Canon 10D: a bizarre CIFF, for which they published the specifications
> for the "superstructure" of this file. It's a kind of weird TIFF; you
> can, in fact, TIFFize a CIFF. There are two images in this file: the
> JPEG and an strangely encoded raw sensor dump. That Dave Coffin
> managed to figure it out speaks for his abilities as a Reverse
> Engineer.
[snip]

By "weird TIFF", do you mean "weird TIFF 6.0" or "weird TIFF/EP"? Is
it, in fact, "standard TIFF/EP"? You may know the following, but I
suspect many people don't:

1. *TIFF 6.0* is owned by Adobe, and was owned by Aldus before that.

When you set your camera to TIFF, that really means TIFF 6.0. Raw
processors normally have TIFF 6.0 as an *output* option. Photo-editors
normally read and/or write TIFF 6.0 as options. (You can actually use
TIFF 6.0 instead of PSD for Photoshop - make sure you ask for layers to
be preserved). If a magazine buys one of your photographs in TIFF form,
they mean TIFF 6.0.

TIFF 6.0 images have the full colour information for each pixel,
(except for monochrome images, of course). TIFF 6.0 is not suitable for
Raw files, because it has no tags defined for holding sensor data, and
there are many other metadata tags that it would need also.

2. *TIFF/EP* ("Electronic Photography") is owned by ISO, and costs 150
euros. It was developed from TIFF 6.0, and much of it is copied
directly from TIFF 6.0.

It has those extra tags for sensor data, (look for tags starting CFA -
Color Filter Array). It also has a lot of extra metadata tags. Camera
manufacturers typically (but not necessarily) start with TIFF/EP when
designing their Raw formats, because it has many of the tags they need.
Indeed, one manufacturer originally used the .TIF extension for its Raw
files.

Like TIFF 6.0, TIFF/EP has very many options. And, of course, although
it is a standard, there is no constraint on manufacturers to conform
precisely to it. So resultant Raw formats proliferate, and there are
lots of unnecessary differences.

3. *DNG*, like TIFF/EP, is a development of TIFF 6.0, but I prefer to
think of it as a development of TIFF/EP. It has a few tags that TIFF/EP
doesn't have, to cater for technology innovations since TIFF/EP was
defined, and for control purposes, such as the 2 version tags that
define the range of DNG versions that a particular file is compatible
with.

DNG cuts through all the unnecessary options in TIFF/EP, eliminates
some of the more complicated structural options, and makes some tags
mandatory. So it ensures that the image data can always be extracted,
along with sufficient extra information that Raw processors can handle
the image data without needing to know about the camera. (The file is
self-contained). But it also defines how camera manufacturers can
safely store any "secret sauce" in a DNGPrivateData tag defined for the
purpose. "Safely" means that anyone else can re-write a DNG file while
preserving the private data, because although its contents are private,
its format is well-structured.

4. To summarise: TIFF 6.0 is what people tend to mean when they say
"TIFF" without qualification. TIFF/EP is TIFF 6.0 plus a lot of the
stuff needed to make Raw files. DNG is TIFF/EP brought up to date and
made fit for purpose. (Now I'll duck for cover!)

--
Barry Pearson
http://www.barry.pearson.name/photography/
http://www.birdsandanimals.info/
Anonymous
May 27, 2005 4:46:09 AM

Archived from groups: rec.photo.digital.slr-systems (More info?)

DoN. Nichols wrote:
[snip]
> But -- I don't see how the situation with RAW files is any worse
> than these limitations from the film days. And in some ways, it is a
> *lot* better. If you process a RAW file with the wrong software, the
> worst that will happen is that you won't get anything useful. The
> original RAW file can be saved until you get the *right* software.

I think the analogy with film can be taken too far. For example, DNG
could be more usefully compared with the developed negative, rather
than the undeveloped negative. Then we would be talking about
standardisation of film sizes to enable the negative to be processed
"anywhere" and by lots of enlargers and scanning and printing
equipment.

For asset management systems that handle many Raw formats, or decades
in the future when we want to process old Raw files, it may be hard to
"get the *right* software".

Sometimes we should just ask "what do we, as photographers or users of
photographs, want?" We surely wouldn't want lots of variants of small
negatives, nearly but not quite 35mm / 2 x 8 sprockets / 24mm x 36mm
image size. That is what we have with Raw formats at the moment.

(I'll pass over the fact that the DNG logo has 2 x 7 sprockets, not 2 x
8!)

> Note that I prefer to do my image processing on unix computers,
> which are *not* supported by either Kodak or Cannon (or any of the other
> digital SLR makers, AFIK). For this reason, I am quite pleased to have
> Dave Coffin's "dcraw" software available.
[snip]

You've just made the case for OpenRAW , and even DNG. Dave Coffin's
work is valuable, but it should never have been necessary. You should
be able to obtain a variety of Unix-based converters, etc, which were
based on proper specifications.

--
Barry Pearson
http://www.barry.pearson.name/photography/
http://www.birdsandanimals.info/
Anonymous
May 27, 2005 5:20:05 AM

Archived from groups: rec.photo.digital.slr-systems (More info?)

In article <d756l8$oih$3@inews.gazeta.pl>,
Alan Browne <alan.browne@freelunchVideotron.ca> wrote:
>RichA wrote:
>
>> http://www.luminous-landscape.com/essays/raw-flaw.shtml
>
>Let's all do our part!!

Well, I've started on my part.

Currently the Adobe DNG converter is pretty much the only choice
if you want to convert your RAW files to DNG. That's not a lot
of help for folks on a Unix/Linux platform, or for the putative
geek 25 years down the road trying to compile on his new platform.

So - I've registered a SourceForge project for an open source
RAW-to-DNG converter. I'd be interested in hearing from C++
programmers who could contribute to the effort.

I'd anticipate spending a week or so collecting contributors,
at which time I'll set up a mailing list and start discussions.
Anonymous
May 27, 2005 5:20:06 AM

Archived from groups: rec.photo.digital.slr-systems (More info?)

John Francis wrote:


> So - I've registered a SourceForge project for an open source
> RAW-to-DNG converter. I'd be interested in hearing from C++
> programmers who could contribute to the effort.
>
> I'd anticipate spending a week or so collecting contributors,
> at which time I'll set up a mailing list and start discussions.

I believe dcraw already reads DNG, if so, adding a write_DNG capability
should be relatively easy.

So considerer Dcraw.c as a beginning for the Guzinta depending on the
copyright provisions. See the GPL statement at top of
http://sunsite.rediris.es/pub/OpenBSD/distfiles/dcraw-7...

From there, given the Adobe spec for DNG, the Guzouta should be
relatively easy.

Perhaps enlist Dave Coffin?

Cheers,
Alan.

--
-- r.p.e.35mm user resource: http://www.aliasimages.com/rpe35mmur.htm
-- r.p.d.slr-systems: http://www.aliasimages.com/rpdslrsysur.htm
-- [SI] gallery & rulz: http://www.pbase.com/shootin
-- e-meil: Remove FreeLunch.
Anonymous
May 27, 2005 7:07:21 AM

Archived from groups: rec.photo.digital.slr-systems (More info?)

In article <d76066$i9d$1@Fuego.d-and-d.com>,
DoN. Nichols <dnichols@d-and-d.com> wrote:
>
> But -- I don't see how the situation with RAW files is any worse
>than these limitations from the film days.

In the film days, you could buy a different (relatively cheap) film
to use in your (relatively expensive) camera. Nowadays the limitation
is embedded in the expensive component, and you can't avoid it.
Anonymous
May 27, 2005 7:15:38 AM

Archived from groups: rec.photo.digital.slr-systems (More info?)

In article <d762d5$9m0$1@inews.gazeta.pl>,
Alan Browne <alan.browne@FreelunchVideotron.ca> wrote:
>John Francis wrote:
>
>
>> So - I've registered a SourceForge project for an open source
>> RAW-to-DNG converter. I'd be interested in hearing from C++
>> programmers who could contribute to the effort.
>>
>> I'd anticipate spending a week or so collecting contributors,
>> at which time I'll set up a mailing list and start discussions.
>
>I believe dcraw already reads DNG, if so, adding a write_DNG capability
>should be relatively easy.
>
>So considerer Dcraw.c as a beginning for the Guzinta depending on the
>copyright provisions. See the GPL statement at top of
>http://sunsite.rediris.es/pub/OpenBSD/distfiles/dcraw-7...
>
> From there, given the Adobe spec for DNG, the Guzouta should be
>relatively easy.
>
>Perhaps enlist Dave Coffin?
>
>Cheers,
>Alan.

Our goals are rather different. Dcraw reads just enough of the various
file formats (DNG included) to be able to produce converted images.

I'm far more interested in getting all that additional metadata from
the RAW files, and storing it in a publicly-documented format.
Anonymous
May 27, 2005 8:13:45 AM

Archived from groups: rec.photo.digital.slr-systems (More info?)

John Francis wrote:
[snip]
> So - I've registered a SourceForge project for an open source
> RAW-to-DNG converter. I'd be interested in hearing from C++
> programmers who could contribute to the effort.
[snip]

I see it. I'll keep an eye on progress.

I don't think I have the up-to-date skills, or the platform, to help
directly. But I would certainly like to trial a Windows version, if
there ever is one, at an early stage.

--
Barry Pearson
http://www.barry.pearson.name/photography/
http://www.birdsandanimals.info/
Anonymous
May 27, 2005 8:36:06 AM

Archived from groups: rec.photo.digital.slr-systems (More info?)

eawckyegcy@yahoo.com <eawckyegcy@yahoo.com> wrote:

> Suppose the above 16/48 column stuff was a Canon "trade secret" they
> would prefer to keep. How could a standard format encompass such
> sensor behaviour without revealing the secret?

Does not the fact that DNG can *already* handle it indicate that this
"problem" is nonexistent? Does not the fact that DNG can fully handle
data from cameras that didn't exist at the time its code was written
mean that this has no actual basis in reality?

--
Jeremy | jeremy@exit109.com
Anonymous
May 27, 2005 8:41:50 AM

Archived from groups: rec.photo.digital.slr-systems (More info?)

eawckyegcy@yahoo.com <eawckyegcy@yahoo.com> wrote:

> Reichman, et al, say they don't want "trade secrets" revealed, but they
> are woefully clueless in that their demands for documentation of these
> files is exactly that.

That's nonsense. No one is asking Nikon for the process by which white
balance is arrived at, only the conclusion. If that could lead to
reverse engineering their process, then the fact that you can get it
using Nikon's software would lead to the exact same eventuality.

--
Jeremy | jeremy@exit109.com
Anonymous
May 27, 2005 9:10:14 AM

Archived from groups: rec.photo.digital.slr-systems (More info?)

Ryadia@Home wrote:
[snip]
> Remove the chemical, temperature and processing time from this
> description and add using proprietary software, the image was
> 'developed' and what precisely is so different about digital photography
> that drives this demand for total disclosure of the process?
[snip]

A problem with this thread is that, because the film and digital
workflows don't have a one-to-one correspondence, people can make
different analogies and come to different conclusions. For example:

1. If we think of a common Raw format as similar to an *undeveloped*
negative, then we might discuss the degree to which development
specifications are open so that many can use them. For example, C-22,
C-41, E-6, K-14. Then we can talk about the advantages of being able to
choose one's service supplier, and perhaps even one's chemical
supplier.

I think it is better to discuss how open the *interfaces* are, rather
than how open the *processes* are. It makes sense for me to ask for a
film to be developed using C-41, but that doesn't mean I need know
about C-41, and indeed the service I use may not have all the details.

Surely we have benefitted from some common specifications in this area?

2. If we think of a common Raw format as similar to a *developed*
negative, then we are more likely to talk about the advantages of
standard film sizes. These make it easier for us to use the same
enlarger, or the same scanner, or slide projector, for many different
makes of film from many different cameras. And we can use many
different services to get others to do this if we don't want to do it
ourselves.

We have certainly benefitted from common specifications in this area.

3. Once we are in the digital realm, we benefit from common file
formats such as TIFF 6.0 and JPEG. A very large set of interchange
benefits have grown up around these standards. I think a common Raw
format is analogous to TIFF 6.0 and JPEG, *not* because they fit into
the workflow at the same place, (they don't), but because they
illustrate what happens when we get recognised common image formats at
various points. Things open up to the benefit of all of us.

I was interested to note that Photobase sell a "Convert to DNG"
service. If they make money from this, it suggests there can be a very
real cost to photographers of the proliferation of Raw formats. I don't
criticise Photobase for offering this service, but it is a pity that it
could ever be necessary.

If we step back from details of the digital workflow, and look at
complex multi-vendor systems of all kinds, we can see many cases where
we get benefits from designing a clean "architecture", with good
specifications for the interfaces between the components. In fact, much
of modern life is build on these principles. Railways: wheel guage;
loading guage; coupling specification; platform height. Computers:
common interfaces all over the place. Bicycles. Guns & bullets. (Hm!)
Credit card specifications.

Why is Raw thought to be so different from just about everything else
in the digital image value chain? Consider:

When I've I've taken some pictures, I put the memory card into a card
reader. The configuration of the pins and the size of the card is
specified by a standard. So is the exact nature of the voltage changes
of the pins as the data is read from the card. The card reader is
plugged into a USB port. USB is a standard specification, of course. My
laptop doesn't have a USB 2 port, so I use a PC Card with USB 2 ports.
The PC Card interface, including pins and voltage changes, is another
standard, of course.

If I shoot TIFF 6.0 or JPEG, they are determined by available
specifications, one a de-facto standard, the other an industry
standard. A magazine might ask for a TIFF 6.0 image. Or if I publish to
the web, a JPEG image will be identified by an HTML page, with the
presentation determined by a style sheet. HTML 4.01 Strict and CSS2 are
"recommendations" (equivalent to standards) of W3C. I upload them to
the website using the standard FTP protocol, and viewers download them
using the standard HTTP protocol.

This systematic coupling of innovative hardware, firmware, and software
components via layers of standards and agreed specifications under
version control is the normal way our complex multi-vendor world works.
But ... I don't shoot TIFF or JPEG. I shoot Raw. Are we supposed to
believe that these principles can't work for Raw formats? That if a
magazine wants my Raw image, I shouldn't give them a standard format,
but I should give them the proprietary specification so that they can
obtain software to extract the image? Of course not! It is obvious that
the same principles can and should apply to Raw formats too.

Raw is not different *in principle* from all the other components. It
simply hasn't yet reached the same level of maturity. Those of us who
realise its immaturity need to be encouraging it to grow up. We musn't
let it believe that it can remain adolescent for ever. We certainly
shouldn't be making excuses for it!

Photographers, and users of photographs, need future Raw formats to
conform to an agreed specification, which will become at least a
de-facto standard. Then we need a way for older Raw files to be
convertable satisfactorily to the agreed format, so that they can be
handled in the same way without risk of being neglected. The primary
reason for obtaining the specifications for those older Raw formats
should be to enable high-quality converters to be developed. NOT so
that all packages that ever handle Raw files recognise all those Raw
formats for ever. Expecting (say) a new Raw processing package launched
in 5 years time to support 100s of Raw formats is just plain silly! Why
should it have to recognise more than 1?

--
Barry Pearson
http://www.barry.pearson.name/photography/
http://www.birdsandanimals.info/
Anonymous
May 27, 2005 10:36:25 AM

Archived from groups: rec.photo.digital.slr-systems (More info?)

eawckyegcy@yahoo.com wrote:
> Alan Browne wrote:
>
>
>>Open RAW is best for everyone in the long term.
>
>
> Be very careful what you wish for.
>

Yes. Mrs Browne wished for a boy with good looks, smart wit and high
intellect. Unfortunately she got Alan!.

Douglas
Anonymous
May 27, 2005 11:41:22 AM

Archived from groups: rec.photo.digital.slr-systems (More info?)

Owamanga (not-this-bit) wrote:
[snip]
> My copy of Word that come in a 2004 version of Office XP still opens
> an ASCII text file format, a standard that dates back to 1963.
>
> Today, the latest version of Photoshop still opens the HPGL/GCG files,
> the oldest graphics format I can think of. Dating back to the early
> 80's or before. Postscript is nearly as old, and again, I can still
> open those.
>
> These are between 25 and 40 years old, and we can still open them. I
> don't think people need to worry *quite* so much.
[snip]

Two points:

1. You name the products that can access those files. In the case of
PDF and ASCII text, there are lots, of course. But is that the case for
HPGL/GCG files? There is a risk that you start to get tied in to
particular products, running on particular platforms, to access these
old file formats.

2. A lot depends on the degree of motivation for products to support
particular formats. ASCII text and PDF are ubiquitous, so it is hardly
surprising that plenty of products can access them. And I think
Photoshop is better than many other products. (Did HPGL/GCG get a good
take-up?)

Can you properly print files created about 20+ years that used an
extension to the WordStar3 dot-commands and special control characters
to draw diagrams on a Ricoh 4120 (?) laser printer? No, of course you
can't! It was "proprietary". I devised that format myself, and wrote a
printer driver for WordStar3 to exploit it. A huge number of documents
in that format are now orphaned. Would it be satisfactory if I simply
provided the specification for those extensions, so that anyone could
write code to access them in future? No! If that was the only way to
save vital documents, it would have to done, but it is hardly the way
anyone would choose.

I have read of a couple of cases where Raw formats have already become
orphaned, or at least lost one parent. (I won't name them in case I
mislead). In one case, a 5-year old camera from a well known
manufacturer was dropped from the latest version of their leading tool,
leaving just their lesser tools to support it.

We have most chance of long life if:
- We have one or just a few formats of that type, instead of 100s.
- We have vastly many millions of files conforming to that format.
Preferably many millions per day.
- We have lots of products that support the format. Preferably
including open source code.

We have that for TIFF 6.0. We have it for JPEG. We need it for a Raw
format.

(I have an uneasy feeling that the camera manufacturers are far less
motivated to support their Raw formats for a long time than software
suppliers are).

--
Barry Pearson
http://www.barry.pearson.name/photography/
http://www.birdsandanimals.info/
Anonymous
May 27, 2005 1:06:17 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

Alan Browne wrote:
[snip]
> Hence my big yawn over DNG ... until I discovered the other day in an
> actual experiment with real files that DNG takes 26 to 41% less space
> than .MRW. So I'm mulling the transition to DNG solely to reduce the
> DVD count by 1 for every 3 or 4 DVD's. I haven't committed to the
> decision yet.
>
> But prior to that, my DNG 'objection' was: when the OEM's adopt it, I'll
> adopt it.

You illustrate why there is so much disagreement over DNG. Whether
someone benefits from DNG depends on the camera used, the workflow, the
version of Photoshop used, what package is used for Raw processing, the
perception of risk and impact of orphaned Raw formats, etc. (The
balance shifts month by month, always towards more likely benefit from
DNG).

If you used the 350D, you would have discovered that the saving was
only about 6% to 7%. (Yawn!) You probably wouldn't even be mulling!

But if you used the Pentax *istD, you would long ago have discovered
that the saving was well over 50% (from about 13MB to about 6MB). And
if, like me, you didn't have a DVD burner, but archived to CD.... You
would already be a very enthusiastic user of DNG!

Here is something I wrote elsewhere:

1. Someone who uses a Raw processor that won't accept DNG will get
little or no benefit from DNG. They may decide to archive a DNG version
of their Raw files to increase the chance that they will be able to
read the files years later. But they will have to work with the
original Raw file. Users of the cameras' own software (currently) come
into this category.

2. Users of ACR 2.x under CS may get significant, or little, or no,
benefit from DNG, depending on their workflow. I benefited, and someone
with a camera only supported by ACR 3.1 / 3.1 DNG Converter, such as
the 350D or D2X, can benefit a lot. But others may not benefit, or only
a little.

3. Users of ACR 3.1 under CS2 are likely to get significant benefit
from DNG. This is the release where its benefits have become much more
obvious. Since upgrading, I have started to convert to DNG straight
from the card without an embedded version of the original file.
(Against the advice of Adobe). Being able to hold ACR 3.1 settings and
adjustments within the DNG file in a non-destructive way makes file
management easier. Obviously I get the benefits of the smaller file
size.

It is not very surprising that a lot of people can't yet benefit, and
lots of people remain to be convinced. DNG was launched 8 months ago
TODAY. That is not long, given the things that have to come together
before most people can benefit. I wonder how many people were talking
about TIFF 8 months after it was launched? PostScript? PDF? HTML? CSS?
(SVG!)

I would advice everyone to investigate DNG, and make up their own mind
about whether it suits them. But, as noted above, it certainly won't
suit everyone.

Yet!

--
Barry Pearson
http://www.barry.pearson.name/photography/
http://www.birdsandanimals.info/
Anonymous
May 27, 2005 1:06:42 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

Jeremy Nixon wrote:

>> Suppose the above 16/48 column stuff was a Canon "trade secret" they
>> would prefer to keep. How could a standard format encompass such
>> sensor behaviour without revealing the secret?
>
> Does not the fact that DNG can *already* handle it indicate that this
> "problem" is nonexistent?

Cite the relevant sections in the DNG specification which detail
exactly this -- that is, how to handle the 16/48 column stuff -- and
you have made a point.
Anonymous
May 27, 2005 1:16:07 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

Barry Pearson wrote:

> By "weird TIFF", do you mean "weird TIFF 6.0" or "weird TIFF/EP"?

I mean "weird TIFF", as "unusual", "not exactly the same as but similar
to in strange way", or "derived from by those under the influence of
unspecified narcotic agents", and so forth. It has a directories,
tags, and the rest of it. Everything else is completely incompatible
with TIFF as is, but it can be easily converted into whatever TIFF one
wishes (if anyone was so inclined).

> But it also defines how camera manufacturers can safely store any "secret
> sauce" in a DNGPrivateData tag defined for the purpose. "Safely" means
> that anyone else can re-write a DNG file while preserving the private
> data, because although its contents are private, its format is
> well-structured.

You need to read the ranting of the OpenRAW people. They explicitly
demand "no secret sauce" -- though irrationally maintaining they are
not making such a demand.
Anonymous
May 27, 2005 3:16:24 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

Jeremy Nixon wrote:

>> Cite the relevant sections in the DNG specification which detail
>> exactly this -- that is, how to handle the 16/48 column stuff -- and
>> you have made a point.
>
> It doesn't have to specify such a thing.

Yes it does. If the native camera format can do things that the DNG
can't, why use the DNG?

> That's not what DNG is, or what it does.

Shifting the goal-posts now? OpenRAW wants full, complete,
documentation.

> It doesn't have to handle, or know about, every technical difference
> in every sensor design.

If it doesn't, then you lose capabilities that may be associated with
the sensor. If the DNG removed the red channels from all RAW images,
would you use it?

> Indeed, it proves that using a different RAW format
> for every camera model is completely unnecessary.

I have data that suggests that Canon cameras have unique black level
estimation for long exposure images. The DNG format does not allow for
such behaviour (or at least you have not presented me with a citation
-- and I couldn't find one when the DNG spec was initially released).

Now if the DNG can't handle that, what do you recommend I do with my
multi-minute minute exposure CR2 files I have?
Anonymous
May 27, 2005 3:44:05 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

Jeremy Nixon wrote:

>> I have data that suggests that Canon cameras have unique black level
>> estimation for long exposure images. The DNG format does not allow for
>> such behaviour (or at least you have not presented me with a citation
>> -- and I couldn't find one when the DNG spec was initially released).
>>
>> Now if the DNG can't handle that, what do you recommend I do with my
>> multi-minute minute exposure CR2 files I have?
>
> It is not something DNG needs to "handle".

Then the DNG is useless re: the above situation, since the native
CR2/CRW files (and their decoders from Canon) handle it just fine.

> Try taking your files, converting them to DNG, and loading them into
> Camera Raw. It'll work.

Something will happen, but probably not the _optimal_ thing. Maybe you
have lesser goals, but I paid alot for my camera, and want my images
given as good a treatment as possible. If DNG can't do it, why should
I use it?
Anonymous
May 27, 2005 3:55:12 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

On Fri, 27 May 2005 20:00:11 +1000, "Ryadia@Home" <ryadia@hotmail.com>
wrote:

<snippage>

>When you didn't like the images from a patented process of developing
>Kodachrome, you simply changed to a different film/chemical combination.
>Same here. Nothing at all to make you stay with Nikon, Canon, Olympus or
>whatever. A lot of noise going on here about nothing.

I guess the point some are trying to make is that with film, if you didn't
like how one flim/developing combination worked, you could pick a different
one but still use the same equipment (camera/lens/falsh etc.) The
difference here, is if you don't like what one camera company are doing,
you've (generally) got to replace the whole kit. It's a far more drastic
change than switching film brands.


Regards,
Graham Holden (g-holden AT dircon DOT co DOT uk)
--
There are 10 types of people in the world;
those that understand binary and those that don't.
Anonymous
May 27, 2005 4:01:00 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

DoN. Nichols wrote:

> If they wanted to be *serious* about the encryption, I don't
> think that "softice" could do much about it.

Cryptographic snake-oil. Numerous people have proposed mechanisms that
can somehow secure information on fundamentally insecure/untrusted
hardware or media. None of them have worked. The RIAA, MPAA and other
"content" people have idiotic, aphysical dreams about this, but it
won't ever happen, no matter how many lawsuis they file, laws they
manage to pass, etc.

> Add a private key to get which you have to send to the vendor
> both an image from your camera (to get the EXIF data), and a unique
> number in the computer (hostID on Sun workstations, something else in
> the firmware on a PC or a Mac). In exchange, you will get a key which
> will work only with *your* camera on *your* computer.

The camera maker _may_ be able to trust the camera, but that's as far
as they can go. Everything else is not under their control.
Extracting the final crypto-key would be a simple debugging job.
Anonymous
May 27, 2005 4:11:11 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

Jeremy Nixon babbles:

> The native RAW files don't have to "handle" it, either.

Once again, nitwit: I have data that hints that Canon has a special
black-level estimator for long exposure images.

This estimator, if it exists, would be built into Canon CR2 file
decoders.

None of this is known to DNG. (I am still awaiting a DNG spec
citation.)

Thus no current DNG decoder uses this special estimator.

Given this context, why should I use DNG over CR2?

Oh, you might argue, but Canon could make a DNG decoder that knows all
about that. Shall I respond to this now, or can you figure out the
problem on your own?
Anonymous
May 27, 2005 4:35:17 PM

Archived from groups: rec.photo.digital.slr-systems (More info?)

Barry Pearson wrote:

<SNP>
>
> But if you used the Pentax *istD, you would long ago have discovered
> that the saving was well over 50% (from about 13MB to about 6MB). And
> if, like me, you didn't have a DVD burner, but archived to CD.... You
> would already be a very enthusiastic user of DNG!

I suggest you become an enthusiastic user of DVD too.
>
> Here is something I wrote elsewhere:

<SNP>

Everyone has to keep updated and abreast of changes, to be sure. But
regarding DNG (or some standard 'RAW') the OEM's really need to navel
gaze a bit and do what's right for the customers, not themselves.

Cheers,
Alan

--
-- r.p.e.35mm user resource: http://www.aliasimages.com/rpe35mmur.htm
-- r.p.d.slr-systems: http://www.aliasimages.com/rpdslrsysur.htm
-- [SI] gallery & rulz: http://www.pbase.com/shootin
-- e-meil: Remove FreeLunch.
!