Category: Technology

HPC on the (relative) cheap using public cloud providers

For the past several years, I’ve been working on leveraging high-performance computing techniques for high-throughput data intensive processing on desktop computers for stuff like image and video processing. Its been fun tracking what the multi-processing end of HPC has been doing, where the top 100 super-computer list has been very competitive and very active. Countries, IHVs and universities vie for who can generate more teraflops; spending millions and millions of dollars on the cooling plants alone for their dedicated data centers. These super computers exist to solve the BIG PROBLEMS of computing, and aren’t really useful beyond that.

At the same time, I’ve been following the public computing clouds like Amazon’s EC2, Google’s App Engine and Rack Space’s Public Cloud. These have been interesting for providing compute on the other end of the spectrum, occasional compute tasks, or higher average workloads with the occasional spike capability (like web apps). The public clouds are made up of thousands of servers and certainly rival or best the super computers in numbers of cores and raw compute power, but they exist for a different purpose.

This article in The Register really got me excited. Especially when I read this:

Stowe tells El Reg that during December last year, Cycle Computing set up increasingly large clusters on behalf of customers to start testing the limits. First, it did a 2,000-core cluster in early December, and then a 4,096-core cluster in late December. The 10,000-core cluster that Cycle Computing set up and ran for eight hours on behalf of Genentech would have ranked at 114 on the Top 500 computing list from last November (the most current ranking), so it was not exactly a toy even if the cluster was ephemeral.

The cost of running this world-class super computer?

Genentech loaded up its code and ran the job for eight hours at a total cost of $8,480, including EC2 compute and S3 storage capacity charges from Amazon and the fee for using the Cycle Computing tools as a service.

Real world HPC is now coming into price points where it is accessible to even small companies or research groups. This seems like a ripe opportunity for companies who can apply HPC-techniques to solve real problems for others, and for tools vendors who can make using these ephemeral clouds easier for companies who want to take advantage of them without having to build up high-end expertise in-house.

A Couple New Pixel Bender Links

Frequent contributor to the Pixel Bender forums, Royi Avital, has released a new set of After Effects and Photoshop plug-ins written with Pixel Bender under the name Flixel Plugins. The first three are now available on aescripts.com

Flixel Plugins on aescripts.com

ApexVJ is a really beautiful Flash-based music visualizer that uses Pixel Bender

Simo Santavirta, the creator, wrote an article on his blog about it.

Moving on…

After many happy and productive years working on Pixel Bender and the Adobe Image Foundation, I’ve decided to take on some new challenges. I’m still at Adobe, but I’m now building a new team and launching a brand new product in the Photoshop family. I can’t say too much yet, but I will have news soon. I’ll still be posting about Pixel Bender stuff here (I’m still a very enthusiastic user!), but for the newest news, you should now also watch the official Pixel Bender blog.

What I’m trying to do in Outlook

So, in my work e-mail, I get around 200 messages a day. I periodically get myself back down to inbox zero, but if I take a day or two off, I immediately get behind. I recently decided on a new mechanism for sorting my incoming mail. First off, would be to divert any mail not sent directly to me (where my name isn’t on the to or cc line) into a separate folder. This would be the stuff I would get to when I had time. Next would be to divert mail where I’m CC’d into a separate folder (this is the mail I’d read after reading my inbox), all mail with me on the TO line would be left to filter to my inbox. This way, I think I could make sure that I’m not losing the important messages in the noise of the stuff that I don’t need to read (but will when I have time).

Unfortunately, Outlook’s rules don’t let me do this. I can create a rule for messages where my name isn’t on the “To” line, and I can create a rule for messages where my name is on the CC line, but then messages where I’m in the CC line get put into two different folders because they aren’t mutually exclusive. Since the rules in Outlook are more or less fixed, there doesn’t seem to be a way to do what I want here.

Any suggestions (other than get a real mail program)?

Apple’s way or the highway

The Daily Show With Jon Stewart Mon – Thurs 11p / 10c
Appholes
www.thedailyshow.com
Daily Show Full Episodes Political Humor Tea Party

Is Apple jumping on the crazy train?

I liked this article on Newsweek: Apple vs. Everybody: The company’s epic battle over a missing iPhone is only the latest in series of contretemps.

It’s the kind of attention that Apple, long a media darling, isn’t used to. Apple’s control-freak nature didn’t matter as much when it was a plucky underdog. Yes, Jobs was a demanding boss and a finicky perfectionist—but he created great products. We rooted for Apple, and wanted it to survive. Apple seemed like the anti-Microsoft, a company that was on our side. But this year Apple will do nearly $60 billion in sales, and its market value stands at $240 billion—the third-largest in the United States, bigger than Coca-Cola and Pepsi combined. Any company that big can seem a little scary. So when police start breaking down doors over a lost phone, it’s a PR disaster, especially for Apple. The company works hard to cultivate a counterculture image, with ads that have featured Gandhi and John Lennon, not to mention the “I’m a Mac” hipster. Yet lately Apple has started to look like the big bully of the tech industry, the kid who doesn’t play well others. Over the long haul, that can put customers off.

It definitely can put developers off (including this one), and when your platform has a lot of competitors gunning for it and a slim percentage of the desktop market, putting off developers is not really a very good idea. Apple is betting the company on their new strategy of a tightly controlled ecosystem where they make a small amount off of every transaction and act as intermediaries between content producers and developers and their customers. It will either be fantastically successful or Apple will crash and burn in a spectacular fashion. Only time will tell.

Section 3.3.1 is not new behaviour from Apple

[disclaimer: I am an Adobe employee and an Adobe and Apple shareholder, my opinions are my own and not those of my employer.]

Like the rest of the software industry, I’ve been pondering what the effect section 3.3.1 of the iPhone 4.0 SDK will have. I had fully been planning to make an iPhone application at some point. I had planned to do the initial version with Flex to prototype, but then also spend time doing a Cocoa version to better learn that SDK for myself. This iPhone 4.0 SDK announcement honestly has me questioning if I do really want to develop for the iPhone. Not just because of a higher-minded sense of indignity at Apple’s lack of openness of their platform, but rather because of that combined with their somewhat arbitrary and opaque app store approval process. Could I spend months of my spare time learning ObjectiveC and working on an iPhone application only to have that time be a complete waste if the App store reviewers decide that they don’t want that app in the store?

Thinking about it this morning, I realized that not only was Apple’s move to lock in developers nothing new, but that I’d already written about it before (in fact, I’ve been blogging about it since almost the day I started doing professional development for the Macintosh): iPhone SDK: The carrot for Cocoa, the stick for Flash, The difference between being an Apple developer and a Microsoft developer, Developers Developers Developers Developers.

Gruber had the motivation right, I think, but I also think he got the ramifications wrong. Since Steve returned to Apple, they have been applying the screws tighter and tighter to their developers, trying to get them to lock in. It was somewhat indirect at first, but the long term implication was clear: “We’ll tell you how to develop for our platform, if you do as we say, then you’ll be fine. If you don’t do it the way we tell you, your life will be a never-ending stream of headaches.” The move to Intel (forcing all developers onto X-Code and a big rewrite of any PPC-assembly) was step one, the move to 64-bit (dropping support for Carbon after promising it) was step two. The iPhone 4.0 SDK is just the most obvious move in this process because it basically spells it out. You no longer have a choice: it is Apple’s way or the highway. The problem is the App store. On the Mac, I control my own distribution. On the iPhone platform, Apple does. That means that they no longer have to negotiate with their developers, they can now finally dictate to them.

As a developer, this makes the iPhone platform a lot less attractive because I also can’t be sure that they won’t change the terms again. Once I’m locked in, I’m locked in. Apple can do whatever they want and I’m forced to rewrite my apps or get forced out. As someone who writes software for a living, this scares the crap outta me.

Here are some other blog posts that I thought were good reading around this:
The iPad isn’t a computer, it’s a distribution channel (O’Reilly Radar)
Five rational arguments against Apple’s 3.3.1 policy (37 Signals blog)

Interesting UX presentation from Theresa Neil

RIA Screen Layouts

View more presentations from Theresa Neil.

The deck is a nice, loose, taxonomy of different design styles (some of the examples seem to have multiple styles to me), the examples are really good and actually pointed out some cool sites that I hadn’t heard about. Definitely worth a flip through if you have an interest in this stuff.

via Ahmed Rabieh

Google vs. Microsoft: Peter Wilson’s view

Peter Wilson spoke at Ignite Seattle last night (How do I keep missing these?!?). Having been a senior dude at both Microsoft and Google, he has some true insider perspective on each. It is through the lens of what he cares about (cloud computing), it does have some good perspective. What is interesting to me as a former softie myself is that while he doesn’t say anything horrible about either place he definitely seems to still be a lot more supportive of his more recent employer than his previous one. While I love the concept of the Ignite talk being limited to 5 minutes, I would have really liked to hear more on this topic. Maybe he’ll do a longer version at some other local tech meet-up that I’ll completely miss as well 🙂

via TechFlash: Google vs. Microsoft: The view from a guy who worked for each

iTunes hack warning!

My iTunes account was hacked on Friday, I didn’t find out until Sunday night though because that was when I tried to log in to my account after it had happened.

Watch out for these signs (Apple support didn’t even consider that I had been hacked when I contacted them):
The hackers changed my account ID and my e-mail address to something similar to what they had been before but different. This allowed them to charge two $50 iTunes gift certificates to my card without me being notified (because they had changed the e-mail address on the account).

When I tried to log into my account on iTunes, I got weird errors about my account id or password being incorrect. When I tried to recover my password on Apple.com, I got an error saying that my account ID was not in the system. When I tried to get my account ID, they couldn’t find it (since the hackers changed my e-mail address). Of course, I could not log into the support site to try to report the issue since it requires my Apple ID. Luckily I found the iTunes store form that let me contact support via the web without logging in and they were able to tell me that I had changed my account ID and e-mail address. I was able to log into the new ID that the hackers made with my old password, which was really lucky since Apple support was clueless about what had happened.

Why didn’t Apple notify my previous e-mail address when the change was made? That is a basic security process that many other sites use. There is a pretty clear pattern of fraud here as well, account information is changed and then large purchases are immediately made. Shouldn’t Apple be looking for this kind of thing?

I have contacted support to find out if they have a process for dealing with fraud, but I can’t help but feel that Apple’s security is somewhat to blame here. I’ll let you know what Apple suggests I do.

I also posted the above on the Apple support site here. Please spread the word so that others aren’t ripped off.

On a side note, I can’t help but wonder how my account got hacked. I’m pretty wary of phishing scams, and that kind of thing. I’m always extremely careful with this kind of stuff. The only thing that occurs to me is that the same day my iTunes account got hacked, I created an account on artaculous.com. I used the same e-mail address (of course) and in this case I was lazy and used the same password as my iTunes account. I generally try to avoid using the same password twice, but it does get hard to remember them all without reusing them sometimes. I have since gone and changed every password on every site that I have accounts on, just in case. I’m not saying that artaculous.com is some phishing scam, but the coincidence is rather odd…

I’ll update this post as I get more info from Apple. Please add a comment if you have heard about this scam or have more information or suggestions.

[Update: 6/29/09, 11:26pm]
Of course, I changed all my passwords on every site I could find an account on today. My e-mail is full of account update notices from a zillion large and tiny companies… Except Apple. I changed my Apple ID (Twice!), my password (Twice!), my security question, my mailing address. Exactly zero messages from Apple letting me know in case it wasn’t me. This really is pretty weak security on Apple’s part.

[Update: 6/30/09 9:09am]
One of the iTunes gifts certificates had been sent to a gmail address. I tried to find a way on the gmail site to let them know that an account was being used or involved in a crime, but couldn’t find a way to do it. Seems weird because I don’t think you need the certificates mailed to you to use them, just the code. The second certificate had not been mailed or had the e-mail addy cleared. Can’t Apple track the IP address of whomever uses the gift certificates to track back to the people who hacked my account? Will they bother? Still waiting to hear back from Apple on letting them know that my account was hacked. Would call their number or try to see a genius, but I’m in jury duty right now.

[Update: 6/30/09 2:04pm]
Apple has responded (excerpt):
I understand you are concerned about purchases that were made with your iTunes Store account without your permission or knowledge.

I know it can be discouraging when fraudulent charges are made on any type account whether it’s your bank or iTunes.

I urge you to contact your financial institution as soon as possible to inquire about canceling the card or account and removing the unauthorized transactions. You should also ask them to launch an investigation into the security of your account. Your bank or credit card company’s fraud department should then contact the iTunes
Store to resolve this issue. The iTunes Store cannot reverse the charges.

Basically, they are pushing this back onto me to deal with my credit card company on. Not overjoyed with this, but fair enough, most people thought that was what they would do. I am a bit concerned that they believe that my credit card could also have been compromised because of this. I thought that my credit card info wasn’t exposed. If credit card info is exposed through iTunes and their security is so lax, I’m going to be wary of giving them any info in the future. I’m also concerned that they aren’t saying that they will do anything to pursue the person who did this. I would like to feel that Apple actually cares about this instead of just blowing it off.

[Update 6/30/09 10:36pm]
Found these links with more info about iTunes account hacks:

[Update 7/2/2009 9:43am]
Apple (correctly) disabled my iTunes account when I reported that it was hacked. They didn’t actually tell me this though, so I didn’t find out until I tried to use it to update my iPhone apps. What did they need to re-enable it? My billing address. Where was my billing address info stored? In my iTunes account. Since that was pretty unlikely to change after a hack, it seems a pretty weak way to verify my identity. I pointed that out in my return mail, but so far iTunes support has ignored all my questions and comments in my messages to them. I guess that is policy, but also a bit lame. I also may have figured out why the hackers didn’t change my iTunes password. I did find a message from Apple in my spam folder notifying me that my password had changed (from when I changed it after getting my account back).

There was enough info in my account that I’ve had to cancel my credit card, and I’m going to need to be extra vigilant for identity theft moving forward. Since then, I’ve changed my payment method to none in the iTunes store. I may have to enter credit card info each time, but that now seems like a minor inconvenience. I have also changed all my other info to be completely bogus so that if someone does hack it again, they won’t have any useful info on me. Why does Apple need by birthday (not birth date for age verification, but birthday)? I’m going to do the same with my other accounts and would suggest it to anyone else concerned about this kind of stuff.

[Update 7/6/2010]
A year later, tons of reports of other accounts being hacked, including several on this blog. Thanks for adding your voices. Meanwhile, Apple has changed NOTHING on their iTunes security processes. They continue to push the blame and responsibility on their customers and the credit card companies. Now, there are reports that iTunes store and account hacking is not only more widespread than has been thought, but also very well organized. When will Apple take some responsibility?

Seriously, set your payment method to None now if you want to avoid having to deal with this pain. It sucks to have to enter in the data every time you make an iTunes purchase, but it sucks a lot less that having to get a new credit card because someone hacked your iTunes account. Trust me on this one.

[Update 12/26/2010]
I was buying some apps on iTunes today. I still keep my payment info set to none and my address set to my non-billing address as a rule and change them when I want to actually buy anything. It is still a massive PITA, but probably helps me avoid dumb impulse purchases 🙂

I noticed something different today, when I changed my payment and address info, I immediately got a message from Apple about the change, and then another one when I changed it back. This is new, and this is good. A trivial change from Apple, and certainly long overdue, but a very positive step.

[Update 3/22/2013]
Apple has now enabled two-factor authentication on Apple accounts. If you are worried about your account being hacked, it would be a good idea to take advantage of this. Here are instructions from lifehacker on how to turn it on.

Server-based DRM solutions are hostile to consumers

I have a long history with DRM (Digital Rights Management): I worked on the Windows Media 7 Encoder team; I worked at two different internet video startups; and as the owner of record label, I experimented with some of the very first paid digital download solutions (all long lost to internet history at this point).

When I first learned about the DRM mechanism where the player would “phone home” periodically to make sure that you were still licensed to the content, I immediately realized that this was a really fragile way to license media. I’m not talking about subscription content (like Rhapsody), streaming media (like Hulu/YouTube/Flash Media Server) or rentals (like Amazon/iTunes rental), I’m talking about content that is purchased by the consumer. The issue is that there are 1000 ways that the user can lose access to their content without any ill intent on their part. This isn’t an issue if the licenser of their content is still in business and supporting the licensing mechanism. However, even large companies sunset their DRM technology support, screwing over their customers (see Google Video and Microsoft Plays For Sure for example). Depending on how onerous the original licensing scheme is and how it was implemented, buying a new computer, changing the hardware configuration, upgrading system software, the company dropping support for the DRM, the licensing company’s servers going down or just the user being without the internet can cause a user to lose access to the content that they paid for and legally own.

Maybe the user got some warning and could back up their content to some other format (if allowed by the licensing scheme, it often isn’t); but maybe they didn’t see or understand the warning. Then it is too late. Is it the consumer’s fault? No, it is never the consumer’s fault. They purchased digital content with the expectation of owning it forever, just like when they purchased their media as hard goods.

Onerous DRM has been put in place by media companies desperate to avoid piracy, but as it has been written about in so many other places, DRM makes more pirates than it avoids. It makes it more difficult for the people who want to get their content legally by adding roadblocks between them and their purchases and it doesn’t stop the pirates who avoid the whole thing. I wonder how many Plays For Sure customers went to an illegal site to re-download the content that they had already purchased when they lost access to it. I wonder if any of them felt like they were breaking the law at that point. I doubt it. They had paid for something and had been denied access to it. Maybe they were mad at Microsoft, but they were probably more mad at the record labels, because that was the product they purchased. Microsoft was just the store.

I was thinking about this again today when I went to purchase a song off of iTunes and found that Apple had lost my Apple ID. This was the Apple ID that I had spent years buying content from iTunes with. Sure, Apple has moved to make their music DRM free, but I haven’t completely updated my catalog yet, and there is a lot of video that I have paid money for as well that is still subject to Apple’s DRM. While their mechanism still allows me to play my content on my authorized computers (as far as I can tell so far), it will not permit me to authorize a new computer. If Apple isn’t able to fix this problem, what happens to the content I purchased over time? If I can’t access it anymore through no fault of my own, am I in the wrong legally to download it off a file-sharing site?

DRM models have continued to evolve over the years, but I think that the audio model has shown the way for purchased content. It is high time for media owners to allow the people that pay for a full copy of their content to own that content outright, with nothing that could prevent the consumer from having access to the content that they paid for, including transcoding as media formats change over time. Otherwise, they will alientate their consumers as they find they cannot have what they paid for.

note: I avoided mentioning the new licensing models that have sprung up, where when you “buy” a copy of a song or movie the license agreement says that you don’t really own it, which is becoming more common as a way to avoid legal issues when user’s circumvent DRM to make fair-use copies or so that they cannot sue if they cannot access their content. I avoided mentioning it because:
A) it muddies the discussion.
B) I think it is evil.