Just wondering….

Yahoo website performance tips

One of the biggest failing junior developers have is that they are too ’smart’. ‘Too smart’??? How can someone be ‘too smart’? Actually pretty easily.

‘Too smart’ is when the person spent hours looking at a problem. And the next day she/he

• realizes the specs would require violating the speed-of-light;
• or has the product manager come up to them and say, “Oh yeah, there was a typo in the spec and it should have said ‘… NOT …’”;
• or the product manager/tech lead (when asked) says, “that’s not what I meant”;
• or the product manager/tech lead (when asked) says, “I didn’t consider that case” (or don’t worry about that case now);
• or is told by a fellow developer, “oh I ran into that problem last week and on the 3rd page of the documentation, a workaround solution to that problem is described.”;
• or discovers they were working on an old version of the code and upgrading to the latest version of code or libraries, solves the problem;
• or finally another developer looks at the problem and points out a simple condition that has been reversed;
• or…

In all cases, the person (doesn’t have to be a developer) violated the 20-minute rule. Quite simply the 20-minute rule says

“If you are stuck for more than 20 minutes, and you are no closer to understanding the source of the problem - then ask for help”

Yes, it could be something ’stupid’ but it could also be something else. Sometimes you will be told to keep on looking because the person in question wants you to learn something. But sometimes there is a fundamental misunderstanding.

So to make sure that you are not asking for help too easily there is the corollary to the 20-minute rule, the 10-minute review rule.

Spend 10 minutes reviewing, how you got to this situation.

• Describe why the previous code is written the way it was. Notice this is not a ‘what’ question. So do not answer ‘what the previous code was doing?’ Answer why did the previous developer did the code this way - do you know how the code is being used? Are you breaking those assumptions?
• Describe why the previous code is now incorrect. List the changed assumptions or requirements.
• Describe what your solution is going to accomplish.
• Describe why your solution is the minimal solution.
• Describe why your solution is the correct choice.
• Describe the single problem that if solved would make your solution work correctly.

At this point, one of these choices is the path that needs to be taken:

• Fundamental misunderstanding about existing code. You realized your initial assumptions were wrong and that your solution is the wrong path. Solution: STOP and look at reevaluate problem discarding old assumptions.
• Missing information. The problem was not specified clearly enough. There are multiple possible problems that are being asked to be solved. Solution: get clarification. Do not proceed until the problem has been clarified.
• Reinventing wheel. Has this problem already been solved somewhere else in the code? If so, refactor so the already created/tested solution can be used.
• Rathole. A simple problem is becoming more and more complex. Are you fixing the problem of the problem of the problem that you started off solving? Are you making changes all over the place? If you are in a rathole, this is a sure sign of Doing-the-wrong-thing. Solution: STOP!
• I don’t know If you don’t know, then you need to ask for help. Solution: Swallow pride and ask for help.

The master list is here.

But just in case it goes away here is the list (pruned for java 6):

Performance Options

Debugging Options

Configuration problems when deploying a new build are high on the “high anxiety” list. The pressure is high to hurry up and get the build deployed. Often times, the deploy happens late at night when the deployer really just wants to go to bed.

The number one way a developer can help out in this situation is very good error messages. If an exception occurs as a result of a misconfiguration: be verbose. Be overly verbose. The sanity you save may be your own.

This error message does not cut it:

Caused by: java.io.IOException: Couldn’t initialize working directory.
at com.amplafi.core.iomanagement.FtpManagerImpl.initializeWorkingDirectory(FtpManagerImpl.java:83)
at com.amplafi.core.iomanagement.FtpManagerImpl.initializeService(FtpManagerImpl.java:50)
… 16 more
Caused by: java.io.IOException: Couldn’t create directory: ftp-working
at com.amplafi.core.iomanagement.FtpManagerImpl.initializeDirectory(FtpManagerImpl.java:61)
at com.amplafi.core.iomanagement.FtpManagerImpl.initializeWorkingDirectory(FtpManagerImpl.java:81)

The error message should have at the minimum contained the following pieces of information:

• The directory name as supplied in the configuration
• The full absolute path to that directory (not the same thing as the configuration name if the name is a relative, not absolute path)
• The reason for the problem (was the parent directory not writable, something already there, disk full, etc.)
• A suggestion on how to correct the problem

The Setup
Before I plunge into my rant, lets review a little-ole documentation. Under java.lang.Object, for equals() we have this:

It is consistent: for any non-null reference values x and y, multiple invocations of x.equals(y) consistently return true or consistently return false, provided no information used in equals comparisons on the objects is modified.

Over in java.util.Collection land, there is this little bit of documentation, highlighting the importance of overriding the default implementations of equals() and hashCode() if one wants to play nice in a Collection:

Many methods in Collections Framework interfaces are defined in terms of the equals method. This specification should not be construed to imply that invoking Collection.contains with a non-null argument o will cause o.equals(e) to be invoked for any element e. Implementations are free to implement optimizations whereby the equals invocation is avoided, for example, by first comparing the hash codes of the two elements.

So for instances of a class to function well as a key in a java.util.Map or be placed into a java.util.Set, the class should override the standard equals() and hashCode() provided by Object. Because of this universal need, it is a reasonable expectation that implementors of equals() and hashCode() pay attention to performance. For example, the programmers who wrote java.lang.String did a fairly simple implementation of hashCode() and then cached the result so the calculation was not repeated for a given String object.

The resulting ubiquity of overridden equals() and hashCode() results in a certain set of expectations:

1. if the object doesn’t change the results of equals()/hashCode() shouldn’t change (otherwise Set collections will break)
2. equals()/hashCode() should be fast and in the case of hashCode - for potentially expensive hashCode() operations the results should be cached.
3. Immutables (such as URL) should be good candidates to be keys in a java.util.Map

The Takedown
So I was completely blindside by the brain-dead, high school, freshman implementation of java.net.URL.

The only thing I will say positive about the java.net.URL implementation equals()/hashCode() is this: buried deep, deep in the documentation, the high school students who wrote the implementation do casually tell you that they are going to screw you over.

In java.net.URL.equals() (only if you go look at the detailed documentation) do you see this:

Compares this URL for equality with another object.

If the given object is not a URL then this method immediately returns false.

Two URL objects are equal if they have the same protocol, reference equivalent hosts, have the same port number on the host, and the same file and fragment of the file.

Two hosts are considered equivalent if both host names can be resolved into the same IP addresses; (emphasis mine) else if either host name can’t be resolved, the host names must be equal without regard to case; or both host names equal to null.

Since hosts comparison requires name resolution (emphasis mine), this operation is a blocking operation.

Note: The defined behavior for equals is known to be inconsistent with virtual hosting in HTTP. (emphasis mine)

Translation: We are going to screw you and you will enjoy it.

The Scream
How does this screw you?

1. Two fundamental operations equals()/hashCode() are now ridiculously expensive since they involve a DNS lookup to see if they resolve to the same ip address. So now an operation that is optimize in other classes is outrageously expensive (milliseconds long) for a very fundamental internet class.
2. URL looks to be an immutable. Immutable objects should mean that the identity: x.equals(x) is true. But this isn’t the case for URL. If you serialize a URL, the resolved hostaddress (it’s transient) is lost. So serialize/deserialize a URL. Wait a little bit. Add a little bit of the Dynamic DNS magic. Presto “http://google.com” will not equal the deserialized version of itself
3. It is simple wrong. Take the case of a hosting service that hosts two different sites on the same server. The DNS lookup resolves the 2 domains to the same physical ip address. URL (with an “assist” from URLStreamHandler, InetAddress, and our very expensive DNS lookup) compares URLs based on the ip address not the entered hostname. As a result, http://my-porn-site.com could ‘equal’ http://jesus-has-saved-my-soul.org. I think someone might disagree with this assessment, don’t you?

The Takeaway
So after these very expensive operations URL.equals()/hashCode() is broken. The implementers of URL clearly decided that they were going to be ‘clever’ and for all their ‘cleverness’ the only thing they managed to do is screw users of this class. Sun should just admit the error of its ways and just reimplement URL.equals() (with hashCode() being the equivalent).

public boolean equals(Object o) {
    if( !(o instanceof URL)) {
        return false;
    } else if ( o == this ) {
        return true;
    } else {
        return ((URL)o).toString().equalsIgnoreCase(this.toString());
    }
}

The old functionality should be quietly pushed to a uselessEquals() method some place.

The Net
Performance problems and bugs can be introduced from the wildest and least expected locations. Always recheck your assumptions.

“When you have eliminated all which is impossible, then whatever remains, however improbable, must be the truth.”
-Sherlock Holmes.

There is a bug (or two) in this code:

    protected void compareStreams(InputStream is, InputStream isFromGet)
            throws IOException {
        byte[] newBuf = new byte[4096];
        byte[] oldBuf = new byte[4096];

        int len = 0;
        while ((len = is.read(newBuf)) != -1) {
            boolean flag = false;
            if (isFromGet.read(oldBuf) == len) {
                flag = true;
                newBuf.equals(oldBuf);
            }
            if (!flag) {
                throw new IOException(”New file differs from old one.”);
            }
        }
        isFromGet.read(newBuf);
        if (isFromGet.read(newBuf) != -1) {
            throw new IOException(”New file differs from old one.”);
        }
    }

Did you spot any problems? Well maybe you didn’t spot any right away. But use a debugger, and the bugs would quickly become obvious.

1. Look at :
   

               if (isFromGet.read(oldBuf) == len) {
                   flag = true;
                   newBuf.equals(oldBuf);
               }
   

   The results check newBuf.equals(oldBuf) is completely ignored. So what happens if 2 streams of the same length but with different contents are compared? This method says they are the same!

2. But even if the check was flag = newBuf.equals(oldBuf), it would still be wrong! Using an IDE (or a debugger) quickly discovers that newBuf.equals():

       public boolean equals(Object obj) {
   	return (this == obj);
       }
   

   This is not the same as comparing the contents of the arrays. The correct call is Arrays.equals(newBuf, oldBuf). And of course after each read the arrays in question should be 0-filled to avoid issues with partially buffers.

3. Next, what happens if the streams are read at different rates? newBuf and oldBuf will miscompared simply because one stream didn’t have data immediately available to it.
4. Finally,

           isFromGet.read(newBuf);
           if (isFromGet.read(newBuf) != -1) {
               throw new IOException("New file differs from old one.");
           }
   

   What is that first line doing in there!? This allows ‘isFromGet‘ stream to be up to 4096 bytes larger than the ‘is‘ stream!

Yecks! So few lines so many major bugs.

Do a line-by-line code review with a debugger, and discover hidden bugs such as these in “perfect” code.

But finally, haven gotten this far… use the (debugged) work of others…

org.apache.commons.io.IOUtils.contentEquals(InputStream, InputStream) is good and correct. Use that.

This was a recent comment on a bug report:

EventImpl stores start and end dates as two strings (milliseconds presentation of date). Not clearly understand how information can be lost.

What is wrong with this statement?

Well nothing is “stored” until there is a persistent representation of the data that will outlast the program being stopped. Period.

Putting some value in an instance’s field does not qualify. Unless and until some storage mechanism (database, serialization to a file, etc.) puts it some place permanent - it is not “stored”.

The developer can only say the value is being stored if:

1. there is an active transaction;
2. the object being modified is mapped to the database tables.
3. the field being modified is mapped to some column within that database
4. the specific object instance being modified is actually participating in the active transaction. There could be an active transaction that this object is not part of!
5. the transaction commits without errors

For the particular bug report, neither the field nor the object was persisted by the database. Therefore nothing was stored.

When it comes to finally blocks, extra care is required. The code within a finally block can make very few assumptions about the state of variables that it is dealing with. Usually where developers slip-up is with null handling. This error is compounded by finally block lacking the try block exception management protection.

An otherwise well-written method with regards to correct try/catch blocks can still fail badly when it comes to the finally block code. A method that handles all exceptions can still throw an exception from the finally block itself. Usually the finally block throws an exception when try or catch block code threw an exception.

This code has this problem:

   private void copyFile(String fileName) throws IOException {
        File destFile = new File(archiveDirectory, fileName);
        FileChannel srcChannel = null;
        FileChannel dstChannel = null;
        try {
            srcChannel = new FileInputStream(new File(workingDirectory,
                    fileName)).getChannel();
            dstChannel = new FileOutputStream(destFile.getAbsolutePath())
                    .getChannel();
            dstChannel.transferFrom(srcChannel, 0, srcChannel.size());
            destFile = new File(archiveDirectory, fileName+".sha");
            srcChannel = new FileInputStream(new File(workingDirectory,
                    fileName+".sha")).getChannel();
            dstChannel = new FileOutputStream(destFile.getAbsolutePath())
                    .getChannel();
            dstChannel.transferFrom(srcChannel, 0, srcChannel.size());
        } finally {
            srcChannel.close();
            dstChannel.close();
        }
    }

Did you spot the problem?

What happens if either of the first two lines throws an exception?

• srcChannel or dstChannel will be null.
• A NullPointerException(NPE) will be thrown from the finally block.
• The NPE will *mask* the original exception.

This last point bears repeating. The original cause of the problem will be lost! Now you as a developer may assume that the lost exception will be an IOException and 99% of the time especially during development this will be correct.

But many developers forget that there are a few externally generated events that can cause throwables to be thrown. Thats right! The root for “things that can be thrown” is java.lang.Throwable - not java.lang.Exception!

Some non-Exception examples include: StackOverflowError and OutOfMemoryError.

So the masking NPE could easily hide an OutOfMemoryError, which results in a lot of hair pulling when:

• the exception is only occurring occasionally…
• only on production boxes…
• at a customer site…
• they will not let you update the code until you have a fix…
• the customer is your company’s largest account…
• and their CEO is calling your CEO…

It may not play out like this in all its glory. But take the words to heart and make sure you never mask Throwables!

Sometimes a code review reveals a code problem that just keeps repeating itself again and again. So rather than creating new email messages I have decided to start posting some code reviews. This will allow me to avoid repeating myself. I can just point to code-review #1 and say read this post.

A few days ago Google announced that they were going to start distributing StarOffice and

Don Dodge is at it again…

What has changed? Star Office has been around for 8 years and has gained no traction. Sun even made it available for free as open source in the form of OpenOffice. That was 7 years ago. Google will be distributing the same StarOffice that Sun now sells for $70, and OpenOffice distributes for free. Oh, BTW, StarOffice is a 200Mb download…or more depending on your OS. [Reality check Don ... how big is Office 2007?]

Why can these PR types ever admit that there is anything to another company’s moves? This is a spin machine answer — it is totally dismissive of this move and completely ignoring what Google is really doing. Google is taking their warchest which is equal to Microsoft and going directly after Microsoft’s cash cow. Google doesn’t have to win this fight - all they have to do is force Microsoft to cut its price and the bleeding will start and never stop. Microsoft while not fading away will certainly take another significant cut in significance.

The good folks over at CNet have it right:

The student version of Microsoft Office runs about $150 per user–and that’s just for Word, Excel and PowerPoint. Google Apps Premier Edition costs about one-third of that, and you get some additional market-proven and reliable collaboration tools like Gmail and Google Talk from a vendor that enjoys tremendous mindshare, good will and mystique. The Microsoft Office 2007 Professional Full Version retails for about $500 per user. You would have to run Google Apps Premier Edition for more than 10 years to make Office a better value.

Don Dodge’s answer:

Microsoft Works is now available for free. Microsoft Works is a fully functional office suite that has shipped on home edition personal computers for years. It compares favorably with any of the new Web 2.0 office competitors.

Sorry Don! Microsoft Works is not ‘free’ it requires Windows! Free is no strings attached. And Windows is not cheap either. And wait a second? You were comparing Google’s offerings to Microsoft’s Office application not MS Works… I sense a bit of a bait and switch.

Back to CNet:

Grill information workplace platform vendors. There’s no reason to run from Microsoft or IBM, just yet, but turn your attention to these two vendors and ask: “OK, so when are you going to offer office productivity software-as-a-service? Or office productivity tools at a price like this?”

Don’s answer for Microsoft is the typical forward-thinking, we-got-a-vision-just-no-product answer:

Software + Services - Microsoft has been working hard to leverage the best of client software and web based services into one seamless user experience. Users want to take advantage of the computing power on the desktop for compute intensive tasks like creating large documents and spreadsheets, multimedia presentations, and video editing. There are other times when you want a more lightweight, collaborative, and interactive environment where a web browser is a better choice. Why not create on the desktop and share on the web? Why not collaborate and edit documents on the web and save the changes back to your desktop?

Lots of great questions … where is the f*ing software? Oh wait, this is what Bill was saying in 2002:

gave Microsoft a grade of “C” for developing what he termed “building-block services” and for making the software-as-a-service concept a reality.

Gates said parts of the software-as-a-service concept are in place in the company’s Hotmail e-mail service. But “a lot is still to be done there,” he said.

Bill Gates in 2005:

Bill Gates pushing Microsoft toward software-as-a-service

The Wall Street Journal has released excerpts from internal Microsoft correspondence in which Bill Gates is calling on Microsoft to jump with both feet into the trend toward software applications being delivered as a service over the Internet.

Whether referred to as Internet services, software-as-a-service (SaaS), or software on-demand, the idea is the same: instead of buying and installing software applications, users simply access such apps over a network. There’s no software to buy. Instead, the application is either paid for on a subscription basis or supported by a third party, such as advertisers.

And now we have Don in 2007….

Creating a great user experience that is intuitive yet powerful, available online and offline, and synchronized across all domains, is the goal of Software + Services. There is lots of work left to do, but that is the direction.

Sounds like what Google is delivering today….while Microsoft still can’t deliver. Maybe Sharepoint is a great thing but from what I hear it is a big intensive, heavy-commitment-required kind of application — which is exactly the opposite of SaaS solutions.

Its funny - microsoft used to make their money on the small companies .. looks like they are trying to be another (failed) enterprise application developer and forgot their roots.