This list is probably far too short. Feel free to suggest additional questions (preferably with answers!)
Octave is probably spending this time recursively searching directories for function files. Check the value of your LOADPATH. For those elements that end in `//', do any name a very large directory tree? Does it contain directories that have a mixture of files and directories? In order for the recursive directory searching code to work efficiently, directories that are to be searched recursively should have either function files only, or subdirectories only, but not a mixture of both. Check to make sure that Octave's standard set of function files is installed this way.