Correct me if I am wrong, but I think that your first generation Core 2 Duo model prefers to have memory installed in equal sized pairs.

"All iMac Core 2 Duo systems have two user accessible slots and may be updated with one memory module at a time. For the ... Login to read more.