100% CPU Utilization, Corrupted Password Files on FMW 11.1.1.2

We are experiencing memory leaks and high CPU utilization for Fusion Middleware (FMW) Version 11.1.1.2  that gradually builds until the server becomes completely unresponsive. The last time this happened rebooting the server did not bring up all of the FMW components.

Errors in the Browser trying to access Oracle Forms:

Failure of server APACHE bridge:
No backend server available for connection: timed out after 10 seconds or idempotent set to OFF.

Errors in the AdminServer.log for Weblogic:

####<May 3, 2011 12:26:38 PM MDT> <Warning> <DeploymentService> <AdminServer> <[ACTIVE] ExecuteThread: ‘0’ for queue: ‘weblogic.kernel.Default (self-tuning)’> <<anonymous>> <> <> <BEA-290014> <Invalid user name or password.>

####<May 3, 2011 12:26:38 PM MDT> <Warning> <DeploymentService> <> <AdminServer> <[ACTIVE] ExecuteThread: ‘0’ for queue: ‘weblogic.kernel.Default (self-tuning)’> <<anonymous>> <> <> <> <BEA-290014> <Invalid user name or password.>

####<May 3, 2011 12:26:42 PM MDT> <Error> <Configuration Management> <> <AdminServer> <[ACTIVE] ExecuteThread: ‘0’ for queue: ‘weblogic.kernel.Default (self-tuning)’> <<WLS Kernel>> <> <> <> <BEA-150035> <An attempt was made to download the configuration for the server WLS_REPORTS by the user with an invalid password.>

####<May 3, 2011 12:26:44 PM MDT> <Error> <Configuration Management> <> <AdminServer> <[ACTIVE] ExecuteThread: ‘0’ for queue: ‘weblogic.kernel.Default (self-tuning)’> <<WLS Kernel>> <> <> <> <BEA-150035> <An attempt was made to download the configuration for the server WLS_FORMS by the user with an invalid password.>

####<Apr 1, 2011 10:58:46 AM MDT> <Notice> <Security> <> <AdminServer> <[ACTIVE] ExecuteThread: ‘4’ for queue: ‘weblogic.kernel.Default (self-tuning)’> <<anonymous>> <> <> <> <BEA-090078> <User  in security realm myrealm has had 5 invalid login attempts, locking account for 30 minutes.>

Searching on MOS came up with two different articles:

Getting Error “User orcladmin In Security Realm Myrealm Has Had 5 Invalid Login Attempts, Locking Account For 30 Minutes.” [ID 1270253.1]

Oracle Middleware 11g – Troubleshooting the Error “Failure of server APACHE bridge” [ID 1304095.1]

Neither MOS document helped as the user was blank in the logs, what user? Notice the space after user in this section: “User  in security realm myrealm”. This is a standalone Forms/Reports install of FMW so there are very few users.

I changed the password for the WEBLOGIC admin account using the GUI Console which really didn’t help. Aha! We had created boot.properties files for starting all services, somehow those were no longer working (corrupt?) I recreated them and restarted all of the services successfully.

For a more permanent fix, we are upgrading to FMW 11.1.1.4 on 64bit RH 5 in a couple of weeks……I don’t recommend using 32-bit OS for FMW.

Advertisements

About April C Sims

Oracle DBA for over a decade...enough said.
This entry was posted in Uncategorized. Bookmark the permalink.

2 Responses to 100% CPU Utilization, Corrupted Password Files on FMW 11.1.1.2

  1. Laura del Riego says:

    Hi April,

    First of all, congratulations for your blog!

    My name is Laura del Riego and I am a community moderator of My Oracle Support (MOS) Communities (communities.oracle.com).

    I have been reading your fantastic blog and I’d like to share it with the rest of customers :-) Do you mind if I reference it in our Oracle Communities?

    As well as this, I’d like to encourage you to publish your blog articles in our MOS
    Communities and thus to increase the value for you and for us.

    Thank you very much and best regards
    Laura del Riego – MOS Community Moderator

    • April C Sims says:

      Laura,

      Go right ahead and reference the blog in the MOS communities. I will start posting them in the communities as I have time. Thanks

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s