About two months ago, we changed our database server from five years old IBM x360 to a new Dell poweredge 6850 with four dual-kernel xeon and 32 G memory and raid5 array. Because we encountered the x86_32 memory limit and the x360 is too old for it's stability.
So we changed from x86_32 to x86_64. the operation system migrated from rhas3 to rhas4(we even used rhas2.1 about seven years ago) x86_64 version, and database still using the Oracle 9i(9208) x86_64, and the not to upgrade to Oracle 10G because our old running applications based on oracle old rbo NOT new cbl.
After the successfull os/database install, i setup the kernel shmmax to 30G, Oracle started successful with default 2G sga, then i changed sga to 26G memory for better performance ,but when startup oracle under sqlplus, it gived me a strange ora-27102 error.
i checked the kernel shmmni,shmall,sem and file-max parameters etc again,nothing goes wrong,strange, except the ora-27102.
after many hours search, the reason raised from deep water. the DEFAULT shmall is 219702 under rhas4, so the system max use memory is 219702*PAGE_SIZE(default 4096)=8G, so when the oracle try to use more than 8G memory, kernel can't permit do this, error ora-21702 then come.
The resolve is sample, changed shmall to 32212254720(30G) under /etc/sysctl.conf, reboot system or /sbin/sysctl -p to let kernel accpet the new setting, then successed to startup oracle with 26G memory.
It's the sample but boring shmall parameter under rhel4 x86_64 kernel.