Another MASSIVE ISSUE with iTunes 11.1

I had thought that the problems with Podcasts in iTunes 11.1 (and 11.1.1) were insane and painful, but I had found work-arounds and had been able to soldier on, until I did a sync of my iPhone today after installing 11.1.1. I got an error that many of my music files couldn’t be synced because they couldn’t be found. Now this was VERY concerning. As I said in my previous post, I have been using iTunes for nearly a decade. I have moved my entire music collection to it and am 100% dependent on it now, syncing several i-Devices. Having podcasts deleted silently was amazingly painful and took days of carefully restoring from backups and re-downloading in order to feel confident that I had gotten everything back. This was incredibly scary. I started to look in my library at some of the files that it had said were missing, and I saw lots of stuff like this:

Screen Shot 2013-10-05 at 1.08.41 PM

That is that the metadata was COMPLETELY INCORRECT. However, this was worse than iTunes messing up it’s own track database. When I went to the disk where the files where supposed to be stored, they were missing. Not in the trash can, GONE.

THIS IS AN INSANELY SERIOUS PROBLEM. Luckily, I have my backups and can restore them, but I can’t use iTunes. At least not 11.1.X. Which means I can’t use iOS7 if I am going to sync my devices. Luckily, I hadn’t done that with most of my devices yet.

Right now I am looking at downgrading to iTunes 10.7 as described here; or iTunes 11.0.5 as described here. I will update if either of those works.

In the longer term, I think I will need to switch off using iTunes to manage my music, podcast and video library completely. At the very least, I will need to keep a shadow library that is separate from what iTunes manages since I can no longer trust iTunes with my files.

[Update 10.5.13]
Was able to downgrade to 11.0.5 successfully and restore the files deleted by iTunes. In the end, iTunes deleted 1750 music tracks; and another 1000 podcasts. I can’t believe that Apple would ship this…

iTunes 11.1 Warning – Major issue with Podcasts [UPDATED 10.5.13 and 3.3.14]

[Update March 3rd, 2014]
I finally was forced to upgrade for work reasons to 11.1.4. I found a suggestion on the Apple forum and decided to try that.

These were my steps:

  • First I backed EVERYTHING up, my media drive and my iTunes library to a separate disk.
  • Then I quit iTunes, moved my iTunes folder out of my user directory so that it wouldn’t get picked up after I restarted.
  • I updated iTunes to 11.1.4.
  • Then I launched the app and let it create a new iTunes folder and library.
  • I made sure that all the sync settings were off, so that no apps or podcasts would be synced over iCloud.
  • I quit iTunes
  • I moved my iTunes folder back into my user directory.
  • I relaunched iTunes and let it update.

So far, this has worked ok (for about 4 weeks for me). I periodically do a check of my library to make sure that no files have been lost and it looks ok for now. I have seen posts on the Apple forum that points to people still having podcasts deleted days after upgrading, so I’m going to continue to check ofter.

I will likely do a similar process every time I update iTunes from now on. I will probably also avoid updating any version as long as I can. Unfortunately, I’ve lost all trust in that application that I have been dependent on for years.

I also want to mention that a friend with contacts on the iTunes team actually forwarded a link to this post and the forum thread as well to some folks in the team. The response (not official, just person-to-person, second hand) was that this wasn’t an issue they thought was affecting many users and therefore it wasn’t a major priority for the team. That may be true (as an engineering leader, I’ve made that decision myself a few times), but as a user it is creating massive problems for, it is of little comfort. This issue may have been fixed by the team anyway, possibly, but the recent comment from Ed, seems to point otherwise.

[Update October 5th – There is a new version of iTunes, 11.1.1, in the release notes it claims that it fixed an issue with deleted podcasts. I installed it. It ran fine for a while (it didn’t fix the podcasts it broke, but it didn’t screw any more up), and then it hung, spinning beach ball. I had to Force Quit it after a few minutes. When I relaunched, it had COMPLETELY REMOVED MOST OF MY PODCAST SUBSCRIPTIONS AND UNSUBSCRIBED ME FROM THE ONES THAT WERE LEFT. Luckily, I had backed up before this happened and I was able to copy over my iTunes folder and relaunch which restored all my podcast subscriptions, until it beach-balled again AND REMOVED THEM AGAIN (I didn’t force quit this time). I then checked my file folders and of course it DELETED MY FILES WITHOUT WARNING, AGAIN!!! DO NOT UPGRADE TO ITUNES 11.1 IF YOU SUBSCRIBE TO PODCASTS! At this point, I once again have to completely reconstruct my podcast library due to poor Apple engineering.]

[Update September 23rd – The Situation is even worse than I thought. iTunes 11.1 is basically useless for podcasts now, see below]

restoring-podcasts

I have been using iTunes since version 1 or 2. I’m not sure. A very long time (nearly a decade). When they added podcast support, I switched from the podcatcher I was using to iTunes and have been using it ever since to sync my podcasts.

While I don’t save every episode from every podcast I have ever had, I do save some of them, which means I have literally years of archived podcasts. Or rather, I should say that I HAD years of archived podcasts. When I upgraded to iTunes 11.1, what I didn’t notice was that Apple somehow unsubscribed me to some of my podcasts or it got confused as to my subscription state. Interestingly, it was the ones that I actually tend to listen to pretty regularly. When it did this, IT SILENTLY DELETED big chunks of the episodes that had been downloaded from those casts.

This is a data-loss bug, the absolutely worst kind of bug imaginable. A stop-ship bug, a never-release-until-fixed issue. Unfortunately Apple did release it. I didn’t notice that this had happened, but at some point, I got a warning about how I was running out of space on my system drive, so I emptied the trash. I noticed that it seemed like I had a lot more files than I expected, but I didn’t think that much about it (I generally leave files in the trash until I need space). A day or so later, I noticed that iTunes didn’t think I was subscribed to a bunch of my podcasts, and that those podcasts were now missing dozens of archived episodes.

So now I will spend the next several days restoring from my on-line and off-site backups and slowly reconstructing my podcast library. Unfortunately, I now also need to worry about what other files may have been quietly cleaned up by iTunes: music, ebooks, movies? If there are more, I may never notice.

In the end, it means that a piece of software that I have used daily and depended on for years and years can no longer be trusted. The effect of this loss of trust cannot be understated. It would be the first step to me looking for another solution; one that wouldn’t have me locked into Apple’s platform. This is why this kind of bug is so amazingly critical to catch and why missing it is not a small issue, but a catastrophic one for an ISV or IHV.

If you are a long-time user of iTunes, beware 11.1, and everyone should have MULTIPLE backups of their files, for just this kind of event. I’m very glad that I have a complete backup of all my files on a hard drive that I can use to restore and an on-line additional backup in case that drive is busted.

[Update September 23rd]

After several hours of re-downloading episodes and restoring from backups, I relaunched iTunes only to find that it had deleted those episodes AGAIN. This means that this wasn’t an issue with upgrading the database, but rather a much more serious issue. This is beyond a critical issue for people who have large libraries of podcasts in iTunes. It seems that it doesn’t affect other parts of the library, but I’m not sure I can trust that for sure. This is a major issue since I have several iDevices and switching to another application is basically out of the question for the moment. I now have to work around this bug and hope that Apple will eventually fix it while being wary of the app deleting files every time it is launched. As a user, this sucks.

Here is the Apple Support forum thread:
https://discussions.apple.com/thread/5322917?start=0&tstart=0

iTunes hack warning!

My iTunes account was hacked on Friday, I didn’t find out until Sunday night though because that was when I tried to log in to my account after it had happened.

Watch out for these signs (Apple support didn’t even consider that I had been hacked when I contacted them):
The hackers changed my account ID and my e-mail address to something similar to what they had been before but different. This allowed them to charge two $50 iTunes gift certificates to my card without me being notified (because they had changed the e-mail address on the account).

When I tried to log into my account on iTunes, I got weird errors about my account id or password being incorrect. When I tried to recover my password on Apple.com, I got an error saying that my account ID was not in the system. When I tried to get my account ID, they couldn’t find it (since the hackers changed my e-mail address). Of course, I could not log into the support site to try to report the issue since it requires my Apple ID. Luckily I found the iTunes store form that let me contact support via the web without logging in and they were able to tell me that I had changed my account ID and e-mail address. I was able to log into the new ID that the hackers made with my old password, which was really lucky since Apple support was clueless about what had happened.

Why didn’t Apple notify my previous e-mail address when the change was made? That is a basic security process that many other sites use. There is a pretty clear pattern of fraud here as well, account information is changed and then large purchases are immediately made. Shouldn’t Apple be looking for this kind of thing?

I have contacted support to find out if they have a process for dealing with fraud, but I can’t help but feel that Apple’s security is somewhat to blame here. I’ll let you know what Apple suggests I do.

I also posted the above on the Apple support site here. Please spread the word so that others aren’t ripped off.

On a side note, I can’t help but wonder how my account got hacked. I’m pretty wary of phishing scams, and that kind of thing. I’m always extremely careful with this kind of stuff. The only thing that occurs to me is that the same day my iTunes account got hacked, I created an account on artaculous.com. I used the same e-mail address (of course) and in this case I was lazy and used the same password as my iTunes account. I generally try to avoid using the same password twice, but it does get hard to remember them all without reusing them sometimes. I have since gone and changed every password on every site that I have accounts on, just in case. I’m not saying that artaculous.com is some phishing scam, but the coincidence is rather odd…

I’ll update this post as I get more info from Apple. Please add a comment if you have heard about this scam or have more information or suggestions.

[Update: 6/29/09, 11:26pm]
Of course, I changed all my passwords on every site I could find an account on today. My e-mail is full of account update notices from a zillion large and tiny companies… Except Apple. I changed my Apple ID (Twice!), my password (Twice!), my security question, my mailing address. Exactly zero messages from Apple letting me know in case it wasn’t me. This really is pretty weak security on Apple’s part.

[Update: 6/30/09 9:09am]
One of the iTunes gifts certificates had been sent to a gmail address. I tried to find a way on the gmail site to let them know that an account was being used or involved in a crime, but couldn’t find a way to do it. Seems weird because I don’t think you need the certificates mailed to you to use them, just the code. The second certificate had not been mailed or had the e-mail addy cleared. Can’t Apple track the IP address of whomever uses the gift certificates to track back to the people who hacked my account? Will they bother? Still waiting to hear back from Apple on letting them know that my account was hacked. Would call their number or try to see a genius, but I’m in jury duty right now.

[Update: 6/30/09 2:04pm]
Apple has responded (excerpt):
I understand you are concerned about purchases that were made with your iTunes Store account without your permission or knowledge.

I know it can be discouraging when fraudulent charges are made on any type account whether it’s your bank or iTunes.

I urge you to contact your financial institution as soon as possible to inquire about canceling the card or account and removing the unauthorized transactions. You should also ask them to launch an investigation into the security of your account. Your bank or credit card company’s fraud department should then contact the iTunes
Store to resolve this issue. The iTunes Store cannot reverse the charges.

Basically, they are pushing this back onto me to deal with my credit card company on. Not overjoyed with this, but fair enough, most people thought that was what they would do. I am a bit concerned that they believe that my credit card could also have been compromised because of this. I thought that my credit card info wasn’t exposed. If credit card info is exposed through iTunes and their security is so lax, I’m going to be wary of giving them any info in the future. I’m also concerned that they aren’t saying that they will do anything to pursue the person who did this. I would like to feel that Apple actually cares about this instead of just blowing it off.

[Update 6/30/09 10:36pm]
Found these links with more info about iTunes account hacks:

[Update 7/2/2009 9:43am]
Apple (correctly) disabled my iTunes account when I reported that it was hacked. They didn’t actually tell me this though, so I didn’t find out until I tried to use it to update my iPhone apps. What did they need to re-enable it? My billing address. Where was my billing address info stored? In my iTunes account. Since that was pretty unlikely to change after a hack, it seems a pretty weak way to verify my identity. I pointed that out in my return mail, but so far iTunes support has ignored all my questions and comments in my messages to them. I guess that is policy, but also a bit lame. I also may have figured out why the hackers didn’t change my iTunes password. I did find a message from Apple in my spam folder notifying me that my password had changed (from when I changed it after getting my account back).

There was enough info in my account that I’ve had to cancel my credit card, and I’m going to need to be extra vigilant for identity theft moving forward. Since then, I’ve changed my payment method to none in the iTunes store. I may have to enter credit card info each time, but that now seems like a minor inconvenience. I have also changed all my other info to be completely bogus so that if someone does hack it again, they won’t have any useful info on me. Why does Apple need by birthday (not birth date for age verification, but birthday)? I’m going to do the same with my other accounts and would suggest it to anyone else concerned about this kind of stuff.

[Update 7/6/2010]
A year later, tons of reports of other accounts being hacked, including several on this blog. Thanks for adding your voices. Meanwhile, Apple has changed NOTHING on their iTunes security processes. They continue to push the blame and responsibility on their customers and the credit card companies. Now, there are reports that iTunes store and account hacking is not only more widespread than has been thought, but also very well organized. When will Apple take some responsibility?

Seriously, set your payment method to None now if you want to avoid having to deal with this pain. It sucks to have to enter in the data every time you make an iTunes purchase, but it sucks a lot less that having to get a new credit card because someone hacked your iTunes account. Trust me on this one.

[Update 12/26/2010]
I was buying some apps on iTunes today. I still keep my payment info set to none and my address set to my non-billing address as a rule and change them when I want to actually buy anything. It is still a massive PITA, but probably helps me avoid dumb impulse purchases 🙂

I noticed something different today, when I changed my payment and address info, I immediately got a message from Apple about the change, and then another one when I changed it back. This is new, and this is good. A trivial change from Apple, and certainly long overdue, but a very positive step.

[Update 3/22/2013]
Apple has now enabled two-factor authentication on Apple accounts. If you are worried about your account being hacked, it would be a good idea to take advantage of this. Here are instructions from lifehacker on how to turn it on.