Pages

Thursday 17 May 2012

Error while starting managed server in OBIEE 11.1.1.5:The persistent file store "_WLS_bi_server1" cannot open file _WLS_BI_SERVER1000000.DAT


While starting managed server we got the error as




weblogic.store.PersistentStoreException: [Store:280105]The persistent file store "_WLS_bi_server1" cannot open file _WLS_BI_SERVER1000000.DAT.
        at weblogic.store.io.file.Heap.open(Heap.java:325)
        at weblogic.store.io.file.FileStoreIO.open(FileStoreIO.java:104)
        at weblogic.store.internal.PersistentStoreImpl.recoverStoreConnections(PersistentStoreImpl.java:431)
        at weblogic.store.internal.PersistentStoreImpl.open(PersistentStoreImpl.java:422)
        at weblogic.store.admin.AdminHandler.activate(AdminHandler.java:126)


Solution:



1. Go to $BI_DOMAIN_HOME


2. find _WLS_BI_SERVER1000000.DAT file as below 


$ find -name "*.DAT"
./servers/AdminServer/data/store/diagnostics/WLS_DIAGNOSTICS000000.DAT
./servers/AdminServer/data/store/default/_WLS_ADMINSERVER000000.DAT
./servers/bi_server1/data/store/diagnostics/WLS_DIAGNOSTICS000000.DAT
./servers/bi_server1/data/store/default/_WLS_BI_SERVER1000000.DAT
./BipJmsStore/BIPJMSSTORE000000.DAT


3.go to the location and then romove it else rename it


$ cd ./servers/bi_server1/data/store/default/
$ ls -lrt
total 1032
-rw-r----- 1 oracle oinstall 1049088 May 17 07:29 _WLS_BI_SERVER1000000.DAT
$ mv _WLS_BI_SERVER1000000.DAT _WLS_BI_SERVER1000000.DAT_old


4. start the managed server as below.


$ cd /orasw/MW5/user_projects/domains/bifoundation_domain/bin
$ nohup ./startManagedWebLogic.sh bi_server1 http://tudevlv0337.us.oracle.com:10000 >startManagedWebLogic.log &




Monday 14 May 2012

Workflow 2.6.4 Configuration In Database 11g Fails With error "WFCA OUT: Error: Could not connect to the remote server."

While installing Oracle workflow in 11g, it fails with error as "WFCA OUT: Error: Could not connect to the remote server. Please check if the server is down or the client is using invalid host, ORMI port or password to connect: Connection refused".

Error in workflow log is as below.
WorkflowCA: Executing: /orasw/app/oracle/product/11.2.0.2_db/jdk/bin/java -jar /orasw/app/oracle/product/11.2.0.2_db/oc4j/j2ee/home/admin.jar ormi://slc00uzk.us.oracle.com:6041 oc4jadmin welcome -application WFMLRSVCApp -installDataSource -url jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=slc00uzk.us.oracle.com)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=B12402Q2))) -username OWFMGR -password ->pwForOwfMgr -className com.evermind.sql.DriverManagerDataSource -location jdbc/WorkflowDS -xaLocation jdbc/xa/WorkflowDS -ejbLocation jdbc/WorkflowDS -connectionDriver oracle.jdbc.driver.OracleDriver

WFCA OUT: Error: Could not connect to the remote server. Please check if the server is down or the client is using invalid host, ORMI port or password to connect: Connection refused
WorkflowCA: Exit Val: 2
WorkflowCA:  Created a redirected data source with application WFMLRSVCApp :
WorkflowCA: Executing: /orasw/app/oracle/product/11.2.0.2_db/jdk/bin/java -Djava.security.properties=/orasw/app/oracle/product/11.2.0.2_db/oc4j/j2ee/home/config/jazn.security.props -Doracle.security.jazn.config=/orasw/app/oracle/product/11.2.0.2_db/oc4j/j2ee/OC4J_Workflow_Component_Container/config/jazn.xml -jar /orasw/app/oracle/product/11.2.0.2_db/oc4j/j2ee/home/jazn.jar -user oc4jadmin -password welcome -adduser jazn.com pwForOwfMgr
The specified user already exists in the system.
WorkflowCA:   Created obfusticated password for redirect datasource:
WorkflowCA: Executing: /orasw/app/oracle/product/11.2.0.2_db/jdk/bin/java -jar /orasw/app/oracle/product/11.2.0.2_db/oc4j/j2ee/home/admin.jar ormi://slc00uzk.us.oracle.com:6041 oc4jadmin welcome -shutdown
WFCA OUT: Error: Could not connect to the remote server. Please check if the server is down or the client is using invalid host, ORMI port or password to connect: Connection refused
WorkflowCA: Exit Val: 2
WorkflowCA:   Executed OC4J Admin script to shut down the OC4J instance :
WorkflowCA: Sun May 13 23:50:13 PDT 2012
WorkflowCA: Workflow Configuration has completed with error.
WorkflowCA: Terminating...

Solution:
Oracle Workflow will not check for or allocate a free port during the installation.By default http-web-site.xml uses port 6004 and rmi.xml uses port 6101. If either or both of these ports are in use by other processes, the Oracle Workflow Configuration Assistant will error out when trying to deploy Workflow Applications. To solve this issue, follow the below steps.


1. Check ports which are not in use as below

#> netstat -atnp|grep LISTEN |grep 6102
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
oracle@slc00uzk.us.oracle.com /orasw/app/oracle/product
#> netstat -atnp|grep LISTEN |grep 6009
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)

and also you can check ports which are not in use as below.

#> telnet slc00uzk.us.oracle.com 6102
Trying 10.241.39.115...
telnet: connect to address 10.241.39.115: Connection refused
telnet: Unable to connect to remote host: Connection refused
oracle@slc00uzk.us.oracle.com /orasw/app/oracle/product
#> telnet slc00uzk.us.oracle.com 6009
Trying 10.241.39.115...
telnet: connect to address 10.241.39.115: Connection refused
telnet: Unable to connect to remote host: Connection refused


2. Make a backup copy of :
- $ORACLE_HOME/oc4j/j2ee/OC4J_Workflow_Management_Container/config/rmi.xml
- $ORACLE_HOME/oc4j/j2ee/OC4J_Workflow_Component_Container/config/http-web-site.xml

3. Modify "port=6101" and "web-site port=6004" to other available ports which can be checked by using the netstat command

4. Re-run Oracle Workflow Configuration Assistant
as below

cd $ORACLE_HOME/owb/wf/install/wfinstall.csh





Thursday 10 May 2012

error ORA-27154: post/wait create failed while starting up database

We got below error while starting up the database.


$ sqlplus '/as sysdba'

SQL*Plus: Release 11.2.0.2.0 Production on Thu May 10 03:14:49 2012

Copyright (c) 1982, 2010, Oracle.  All rights reserved.

Connected to an idle instance.

SYS@BI2400R1 AS SYSDBA 10-MAY-12> startup;
ORA-27154: post/wait create failed
ORA-27300: OS system dependent operation:semget failed with status: 28
ORA-27301: OS failure message: No space left on device
ORA-27302: failure occurred at: sskgpcreates

Solution: This is caused by Kernel parameter setting. Since we were running many databases on a server, we should check our semaphore. Recommended value is

# semaphores: semmsl, semmns, semopm, semmni 
kernel.sem = 250 32000 100 128

In our case it's 

$ /sbin/sysctl -a | grep sem
error: permission denied on key 'kernel.cad_pid'
kernel.sem = 250        32000   100     142

and still we were getting the error as more than 15 databases were running on the server.

We increased the value to 275 as below

# /sbin/sysctl -w kernel.sem=275
added this new value into /etc/sysctl.conf in order the changes persist after system boot
  
and then started the DB.

$ . oraenv
ORACLE_SID = [B12400D1] ? BI2400R1
The Oracle base remains unchanged with value /orasw/app/oracle_test
$ sqlplus '/as sysdba'

SQL*Plus: Release 11.2.0.2.0 Production on Thu May 10 03:18:24 2012

Copyright (c) 1982, 2010, Oracle.  All rights reserved.

Connected to an idle instance.

SYS@BI2400R1 AS SYSDBA 10-MAY-12> startup;
ORACLE instance started.

Total System Global Area 1603411968 bytes
Fixed Size                  2226912 bytes
Variable Size             788530464 bytes
Database Buffers          805306368 bytes
Redo Buffers                7348224 bytes
Database mounted.
Database opened.
SYS@BI2400R1 AS SYSDBA 10-MAY-12> exit