Update 2017: By now it’s clear that an equal amount of JS-bytes and JPG-bytes are not the same. The JS still needs to be evaluated and executed, which also comes as a cost.
People often build beautiful sites with multiple easy-to-use JavaScript libraries. Then, when it comes to addressing frontend performance, suddenly those libraries are an enormous download that the users are forced to bear.
Before you go worrying about how to minify every last library or shave tests out of Modernizr, try and see if you can remove just one photo from your design. It will make a bigger difference.