The Mechanism of Apple's DRM
 

From the 'Fairplay' thread which just got shutdown I was enjoying the discussion on how Apple may be applying their *DRM* strategy. Whether you're a believer in *DRM* or not, I'm just incredibly curious about the mechanism and I thought we could have an open discussion about it. I'm not interested in cracking their technology just trying to understand where this may lead in the future for other content delivery systems on our Macs.

From what I have read so far, they must be taking a unique identifier for the machine, (either serial number or MAC address) hashing it with the AppleID and then writing a key into the .m4p (protected) AAC encoded file itself. Kind of like PGP's public and private key system. Apple retains the private key and you get the public key.

Then if you burn it to CD the resulting .m4a AAC file does not contain a key, which is how you can listen to it on any device.

The new iPods (and the firmware update for the old ones) may gain the ability to deal with .m4p, whereas most other devices can't (yet).

My question: does the iPod support .m4p, or are the files transferred .m4a?? I don't have an iPod, so I can't try this.

It's logical that they must be transferred .m4p to iPods. Otherwise the protection mech would be too easy to break - the industry wouldn't like that!

Anyone have the new iPod yet, to test?

Craig, that's an elegant enhancement of what petey, hayne and I discussed on the last thread.
If the m4p file is decompressed to AIFF format on a CD the key is removed, but if you try to rip it back to MP3 or m4a the sound quality is supposed to drop. Is that correct? If it is then the AIFF file from the m4p must have some data embedded in it to cause this 'corruption'.
The confusing part is the iPod stuff, they probably handle m4a. In the transfer process the 'key' is stripped and the file may be converted to m4a. You could check this via terminal and look for the file extensions.
If any iPod you own can keep and play those files then validation has to be dropped somewhere. If you manage your own playlists then you can simply drag m4p files over to your iPod, or any iPod right? You can't drag them off the iPod library back to your computer. I wish I could "test" this.

edit: Jacques, I don't think you need a new iPod for this test, the new firmware update should enable all iPods, I think. you're right about protection mechanism, in retrospect, iPod Viewer or some other program could just open up the files and allow transfer to any computer.

bassi,

my guesses:

- any iPod will play any M4P.

- you can only transfer an M4P to an iPod from an authorized computer.

- (i'm really guessing here, but i bet the above item only applies to doing the transfer using iTunes. i imagine 3rd party tools will transfer any M4P to any iPod from any computer.)

- the file remains M4P on the iPod, so if you use a 3rd party tool to transfer the M4P from the iPod to a different computer, the DRM remains in the file.

- the quality loss in transcoding AAC to MP3 has nothing to do with the DRM. it's inherent in recompressing any format. going M4A to MP3 would have the same issues.

I believe the loss in quality that has been refered to is merely the usual result of re-compressing a file that was expanded from a different compressed form. Each of these lossy compression schemes (AAC, MP3) drops some information from the original and each scheme works well (gives quality sound) only when the original is pristine.

So if I were to convert an AAC file to AIFF for a CD, then rip that AIFF back to AAC at the same bit rate etc. of the original file, the quality of the new AAC should be lower. Is that correct? If the same engine is being used to compress the file and 'expand' it to AIFF the m4 files should be identical, talking from a mathematical standpoint. This may be an old argument, I don't know if it is.

From a m4a and m4p standpoint it might be important.

I don't know about identical, but I think it is correct that the transition AAC->AIFF->AAC won't lose much, but AAC->AIFF->MP3 will lose significantly in quality.

Here's an analogy:
Text compression method AOA saves space by omitting all the a's and o's in the text (with the idea that you can still understand the text even without the a's and o's) and then compressing further by some other non-lossy algorithm.
Text compression EEE saves space by omitting all the e's in the text (with the idea that you can still understand the text even without the e's) and then compressing further by some other non-lossy algorithm.

She sells seashells by the sea shore
AOA -> She sells sehells by the se shre
-> uf2wheuifwqfoiufh


uf2wheuifwqfoiufh
reverse AOA ->She sells sehells by the se shre
followed by EEE -> Sh slls shlls by the s shr

You get the idea.

Assuming your actions are legal..

Going from the MP3/AAC/Ogg burn to analog and then ripping back to compressed digital may yield better results.

Hmm?

Maybe I didn't make myself very clear on this. What Jacques said is right, that is the flow of information I was originally talking about.

If it's true that an m4p file, when decompressed and burned on to a CD as an AIFF file, cannot be recompressed back to m4a without loss of integrity, then the file being burned on the CD is not a true AIFF file. It's AIFFp.

I'm not talking about converting it back to MP3 at all. Just the AAC format for now.

m4p >> AIFF >> m4a

Am I missing something here.

[I don't want to copy the m4p files and redistribute them, I live outside the US so I can't do it if I could. Just curious]

loss of data upon compression
 

I don't think there is any such thing as an AIFF file that will not lose integrity upon lossy compression. AAC is a lossy compression method. While it may be true that if you go from
AAC-1 -> AIFF-1 -> AAC-2
there is relatively little difference between AAC-1 and AAC-2, I think there is always going to be some small difference. This difference is nothing to do with anything special (your 'p') in the AIFF file, it is merely a fact of life with lossy compression.

In other words I don't think there is some subset of AIFF files (the ones you create by decoding from AAC files) which undergo non-lossy compression. I think there is always some loss, but the amount of loss is less for some AIFFs than others.

Re: loss of data upon compression
 

hmm, i've got no proof, but i just don't buy this AAC-1 -> AIFF -> AAC-2 as not losing a lot of quality. it's like a copy of a copy. an interpretation that has been scrubbed and cooked twice.

here's the start of the experiment, a three minute song:

I'm Eighteen 2:58 Alice Cooper Alice Cooper's Greatest Hits Rock

$ file /Volumes/Alice\ Cooper\'s\ Greatest\ Hits/1\ I\'m\ Eighteen.aiff
/Volumes/Alice Cooper's Greatest Hits/1 I'm Eighteen.aiff: IFF data, AIFF-C compressed audio

$ ls -s /Volumes/Alice\ Cooper\'s\ Greatest\ Hits/1\ I\'m\ Eighteen.aiff
30771 /Volumes/Alice Cooper's Greatest Hits/1 I'm Eighteen.aiff


okay, so there's a .aiff file, from the CD, i presume in it's pristine format, 30MB large.

now then, rip it to AAC, and you get what?

$ file /Volumes/flivver/cd_Ripper/Alice\ Cooper/Alice\ Cooper\'s\ Greatest\ Hits/I\'m\ Eighteen.m4a
/Volumes/flivver/cd_Ripper/Alice Cooper/Alice Cooper's Greatest Hits/I'm Eighteen.m4a: data

$ ls -s /Volumes/flivver/cd_Ripper/Alice\ Cooper/Alice\ Cooper\'s\ Greatest\ Hits/I\'m\ Eighteen.m4a
2844 /Volumes/flivver/cd_Ripper/Alice Cooper/Alice Cooper's Greatest Hits/I'm Eighteen.m4a

now, burn it to an Audio CD and you're going to get back a ~30MB .aiff file (???), with a lot of missing data, i call "air".

now, if you then encode that CD track to ACC or what have you, wouldn't it repeat it's interpretation on the file? not knowing that it was inflated from an AAC source, or any other source?

so, we have the data washed and cooked twice now, introducing more interpretation, even of the air we created?

i can't complete the experiment, can't create this Audio CD for the test at this time.

but i would like to see/hear results of the AAC to AIFF back to some compressed audio format.

lossy! timmy's trapped in the well!
 

bassi, the correct analogy is generational loss in making copies of cassette tapes. everytime you compress into a lossy codec, you are going to lose information. go AAC -> AIFF -> AAC -> AIFF -> AAC enough times, and you'll end up with some form of white noise, the same as if you made a copy of a copy of a copy of a cassette enough times.

there seems to be a widespread impression that AAC -> AIFF -> AAC will provide better quality than AAC -> AIFF -> MP3. as far as i can tell, this impression is mistaken. i haven't performed tests, but i do have some experience with codec issues, and i've never run across the particular piece of wisdom that recompression with the same lossy codec produces better results than using a different lossy codec.

and i'm with mervTormel that the AAC -> AIFF -> AAC conversion will produce rather noticeable degradation.

however, the definition of "noticeable" depends on the sensitivity of the ear of the listener. when i watch Digital Cable TV, i constantly notice motion artifacts from the low bitrate MPEG-2 compression they use. my girlfriend can be sitting next to me and not see any of it. neither one of us is wrong, we just have different thresholds for when the degradation is visually noticeable.

with that disclaimer said, played side by side, i think AAC -> AIFF -> AAC degradation will be noticeable by almost anyone who isn't legally deaf.

the real issue is whether or not the recompressed AAC will be of acceptable quality. for me, i'd easily say no. for some people, it should be acceptable.

thanks, petey. i will look/listen for corroboration.

considering the music i prefer, i am illegally deaf :D

no, mervTormel. you're deaf if you DON'T like alice cooper. in fact, i believe that is the actual legal definition of deafness. (killer is his best album, y'know...)

ha! yeah. killer slays me.

Hayne, mervTormel and petey, thanks for the clarification.

mervTormel's experiment is worth a try but subjectivity is another key. Need to do a double blind test on this.

For what it's worth, I just ripped a song from a store bought CD into AAC at 128 kbps/160 kbps and 192 kbps. The differences between 128, 160 and 192 are noticable, but below it all is the washed out feel of the lower frequencies compared to my normal VBR ~224 kbps MP3. I wonder if they filter out the frequencies below 10 hz to save space. I want to feel the brown note :eek:

Personal taste aside, the main question still remains. What has Apple done to the m4p files and how do they manage the DRM w.r.t. our Apple ID etc. The answers so far are quite good and I imagine only Apple can give the skinny on this. More conjecture and hand waving needed here, just for fun.

Plus, Craig's question about the format of the m4 files from the music store on the iPod is worth answering. Anyone?

bassi,

as related previously, here's my educated guess:

- the files are transferred to the iPod as M4P.

- any iPod can play any M4P.

- the DRM comes in because iTunes will only let you transfer an M4P to an iPod if it's authorized on the transferring computer.

the work around would be to use a 3rd party app to load up your iPod with M4P's traded through the net. this fits in with the whole Fairplay idea of making piracy too much trouble for most people, but not absolutely impossible.

i arrived at this because no other scheme seems to work. i don't think apple can store the files as M4A on the iPod, because they'd be too easy to access and then trade across the net. i'm guessing apple would prefer any net trading be in M4P, rather than M4A.

and since they're not making people authorize iPods, i can't see how they can prevent one of the two forms of trading.

anyone see anything different than this?

bassi,

one more thing: apple WON'T be giving the skinny on how this works.

the first rule of DRM is don't talk about DRM.

petey, I can understand why the first rule is necessary, from a business standpoint. From a user standpoint I just don't like it.

As an analogy, I just went to see a movie and paid around $9 to get in. I sat through 20 minutes of adverts which was really annoying because I don't like to pay $9 to watch advertising and a movie.

(note: one can argue that the movie itself may be advertising of one sort but that's a whole other issue)

From what I can tell, when you authorize a computer, you are authorizing an account, not a song. I can authorize one song on my eMac, unplug it from the network, copy another song bought with my account to it, and it will play both songs. If I deauthorize the computer, it will deauthorize for all songs with that account.
Your apple music store account name and ID are stored in the m4p file, and you can see them if you get info on a file using iTunes. Also, QuickTime and the Finder behave according to the rules of authorization, which can only be controlled through iTunes.
I would guess that Apple is storing the ethernet ID (MAC address) of an authorized machine with your account on their servers, and the machine gets access to songs of that account, which is stored on the machine. But, that doesn't quite make sense, as you can use a modem, airport, or ethernet to connect to the net. AirPort and Ethernet have different MAC addresses. Also, if your motherboard goes kaput, and it gets replaced, your MAC address changes. This brings up the fact that they may be using serial numbers, or a combination of things (most likey, in case of some component getting changed or replaced), but that brings me to "where is the SN on the machines stored?". My guess would be logic board, but a more logical place would be the Mac OS ROM file. I would hope that Apple thought of the possibility of a component going bad. Also, I have personally switched quite a few components on the colored iMacs, and yet firmware upgrades and OS9 system restores (background color) "know" what color the machine is; so I almost wonder if there is an identifier in every machine that Apple doesn't tell us about.
I would be curious to know how this part of the system works, from a purely technical and curious standpoint. I have an unquenchable need to know how things work. I have no intention to try to crack Apple's DRM, as I believe their implementation is almost perfect (not being able to rendevous stream m4p files to unauthorized machines bugs me, what if i have 4 macs?), and am pretty happy with what they have done with it. I just flat-out want to know how it works. ;)

hmm, serial numbers in the ROM. wasn't that the cause of quite a stink from Intel technology?

i wonder if this magic number is a hash of several gestalt values intrinsic in every rig. then, of course, migrations to new iron in the future are going to cause a huge headache to "re-own" property that you've bought.

and then, there's my identity etched in it, and someone is tracking that i buy esoteric music, so why not market me other esoteric things (shudder at the thought of what some mktng shill thinks i would think is esoteric).

i think i'm gonna continue to shop and buy my merch the old fashion way until they pry my cold dead fingers from my little green rectangles.

When I say the Mac OS ROM file, I'm referring to the file that gets loaded into RAM on boot, after initial startup, to finish the boot process. The "ROM in RAM" boot. Not the actual ROM chip on the board.
I believe they are at "/System Folder/Mac OS ROM" and "/System/Library/CoreServices/BootX"

edit
Almost forgot, Apple says to deauthorize a machine before selling it, so I would guess that would mean that when "mirgrating to new iron", it would be best to deauthorize an old machine before nuking a drive for sale.

yeah, i grok, but serializing the ROM file would be folly, because it can be distributed. right?

yeah, deauthorize a rig on sale, or before implosion :D but, still, it's another headache maintenance step in owning a product. tangible, hands on, "i own it" kinda stuff, that you don't have to speak to another entity to validate with your mother's secret recipe.

I think the ROM file idea might be possibility but opens it up for reverse engineering and a whole host of problems (like the distribution).

mervTormel pointed out something interesting. Apple has created this DRM tech and will compile a slew of info based on individual music habits. All in a neat encrypted file database we have no idea how they use or mine. They could go the Amazon route and allow .Mac users who have the same musical tastes to 'discover' each other. Collaborative filtering. Would that be cool? They could then attempt to share their files. ;)

I want to know more, unlikely, I'm sticking to independent buying for now. Well, I have no choice anyway until they expand to these shores. If the mechanism is foolproof why not make it transparent for the masses. True hackers will probably be publishing the details soon, wait and see.

I don't understand the widespread speculation on how Apple's recognizes a particular machine...MAC address, a hash? Do you forget that every Mac has a unique serial number? You see it when you run Apple System Profiler. That's the most logical number to use.

I think all they did was add a tag with your ID and a key. You can open the file with a text editor and see it. I don't think the audio is changed in any way. It's just that all of the Mac software that plays MP4 files checks the key first.

Chris

The serial number as a key did come up in this thread. It got misplaced :)

Being based on serial number makes the most sense, as it is a unique identifier to the machine, and not a particular component.
I never have bothered to check the software checkable serial number of a machine after swapping it's logic board, to see if it is stored there or not.

i know for a fact that, many moons ago, i frob'd this rig with TechTool Lite, "Zap the entire PRAM chip", and the customer serial number and mfg date went missing. now, ASP reports a new serial number and sales order number. hmm?

Very interesting indeed. I'm guessing that since you say "new serial number", it doesn't match the one on the back/bottom/whatever of your machine.
If it is the same, then I would think that the serial number is kept in multiple places, and if one is wiped clean, the others refill it. If so, this could also lend a theory to why imac restores and firmware updates seem to know the color of the machine, even after part swapping.
Also, on the newer machines, doens't booting into open firmware and doing a "reset-nvram" set the NVRAM back to a factory state, and deep zap the PRAM? If so, I did this on an 800Mhz iBook yesterday, and the serial in ASP held.

Maybe if I get into a 1:00AM caffeine induced state again, I'll dig further into m4p files, file modification dates, and what happens when you authorize and deauthorize a machine. If I'm going to do that though, I'm going to need to find my packet sniffer, and my copies of resourcer and hexedit. ;)

On the degradation of quality: it seems apple is ripping it's AAC from the masters (not from cds) therefore the codec has more info and renders a better sounding file; it seems that's why their 128kbits sound better than a rip you'd do on your own.

What's the difference between m4p and m4a? Is it only the presence of a key?


v