How to add second forms node as Forms metric client on 11i (11.5.10.2)

Since we have 32 bit Application servers on PROD EBS 11i, we have bottleneck on memory. There were two options on the table, shared appl_top and second forms node as metric server-client.
We choose latter because shared appl_top needs shared file system which brings more complexity.

In order to user forms metric server-client option, you should first "add a new node" to your EBS environment.

We used cloning method to add a new node. (Doc ID 230672.1 Section 4.4)

• Run preclone on application node (the single node  to be cloned)
• Clone this server with Acronis or VM clone if it is on a virtual server or install a new OS with required packages and copy all EBS files)
• Change IP, hostname on the new cloned node. (If it is cloned with VM or Image)
• Run on second node (new clone):

[appPROD@erpapp1f bin]$ perl adcfgclone.pl appsTier

Enter the APPS password [APPS]:

apps

First Creating a new context file for the cloned system.

The program is going to ask you for information about the new system:

Provide the values required for creation of the new APPL_TOP Context file.

Do you want to use a virtual hostname for the target node (y/n) [n] ?:

Target system database SID [PROD]:

Target system database server node [exafkmdb01-vip]:

Target system database domain name [xxx.com.tr]:

Does the target system have more than one application tier server node (y/n) [y] ?:

Does the target system application tier utilize multiple domain names (y/n) [n] ?:

Target system concurrent processing node [erpapp1]:

Target system administration node [erpapp1]:

Target system forms server node [erpapp1]:

Target system web server node [erpapp1]:

Is the target system APPL_TOP divided into multiple mount points (y/n) [n] ?:

Target system APPL_TOP mount point [/u02/PROD/PRODappl]:

Target system COMMON_TOP directory [/u02/PROD/PRODcomn]:

Target system 8.0.6 ORACLE_HOME directory [/u02/PROD/PRODora/8.0.6]:

Target system iAS ORACLE_HOME directory [/u02/PROD/PRODora/iAS]:

Do you want to preserve the Display set to erpapp1:1.0 (y/n) [y] ?:

Do you want to preserve the port values from the source system on the target system (y/n) [y] ?:

RC-50220: Warning: Java Object Cache Port: 12345 is not free. Unable to preserve the port settings from source system.

Clone Context uses the same port pool mechanism as the Rapid Install

Once you choose a port pool, Clone Context will validate the port availability.

Enter the port pool number [0-99]:

0

Checking the port pool 0

done: Port Pool 0 is free

Complete port information available at /u02/PROD/PRODappl/admin/out/PROD_erpapp1f/portpool.lst

Verifying Database Connection...

Database Connection on Port 1521: Successful

UTL_FILE_DIR on database tier consists of the following directories.

1. /usr/tmp

Choose a value which will be set as APPLPTMP value on the target node [1]:

Backing up /u02/PROD/PRODappl/admin/PROD_erpapp1f.xml to /u02/PROD/PRODappl/admin/PROD_erpapp1f.xml.bak

Creating the new APPL_TOP Context file from :

  /u02/PROD/PRODappl/ad/11.5.0/admin/template/adxmlctx.tmp

The new APPL_TOP context file has been created :

  /u02/PROD/PRODappl/admin/PROD_erpapp1f.xml

Log file located at /tmp/CloneContext_0108020115.log

Running Rapid Clone with command:

        perl /u02/PROD/PRODcomn/clone/bin/adclone.pl java=/u02/PROD/PRODcomn/clone/bin/../jre mode=apply stage=/u02/PROD/PRODcomn/clone/bin/.. component=appsTier method=CUSTOM appctxtg=/u02/PROD/PRODappl/admin/PROD_erpapp1f.xml  showProgress contextValidated=true

Beginning application tier Apply - Fri Jan  8 14:03:43 2016

Log file located at /u02/PROD/PRODappl/admin/PROD_erpapp1f/log/ApplyAppsTier_01081403.log

                                          

Completed Apply...

Fri Jan  8 14:05:52 2016

Beginning APPSIAS_PROD registration to central inventory...

ORACLE_HOME NAME   : APPSIAS_PROD

ORACLE_HOME PATH   : /u02/PROD/PRODora/iAS

ERROR: Registration Failed... Please check log file.

You can rerun this registration with the following script:

        /u02/PROD/PRODcomn/admin/out/PROD_erpapp1f/regOUI_APPSIAS_PROD.sh

Starting application Services for PROD:

Running:

 /u02/PROD/PRODcomn/admin/scripts/PROD_erpapp1f/adstrtal.sh APPS/<appspwd>

• After adcfgclone.pl completes, Source the Applications environment and run the following commands on the Target System:  

·       

   [appprod@erpapp1f bin]$ perl adaddnode.pl

Enter the APPS username: apps

Enter the APPS user password: 

        Using CONTEXT_FILE : /u02/prod/prodappl/admin/prod_erpapp1f.xml

        Using      CTXORIG : /u02/prod/prodcomn/clone/bin/../context/apps/CTXORIG.xml

APPL_TOP name of the source system (old APPL_TOP name) : erpapp1

APPL_TOP name of the target system (new APPL_TOP name) : erpapp1f

Configuring snapshot information for the original (clone source) node...

Executing command:

sqlplus /nolog @/u02/prod/prodappl/ad/11.5.0/patch/115/sql/adadmrat.sql APPS apps prod prod erpapp1f erpapp1 /u02/prod/prodcomn/temp/adadmat_sql.spool

SQL*Plus: Release 8.0.6.0.0 - Production on Fri Jan 8 14:12:12 2016

(c) Copyright 1999 Oracle Corporation.  All rights reserved.

Connected.

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Current data

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

APPL_TOP Names in AD_APPL_TOPS:

*PRESEEDED*

GLOBAL

ap302app

testapp5

erpapp1

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Updating tables...

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

PL/SQL procedure successfully completed.

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Updated data

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

APPL_TOP Names in AD_APPL_TOPS:

*PRESEEDED*

GLOBAL

ap302app

testapp5

erpapp1

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Done

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Commit complete.

Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production

With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,

Data Mining and Real Application Testing options

Completed snapshot configuration on source node.

Configuring snapshot information for the new (clone target) node...

Executing command:

sqlplus /nolog @/u02/prod/prodappl/ad/11.5.0/patch/115/sql/adadmdat.sql APPS apps prod prod erpapp1 erpapp1f /u02/prod/prodcomn/temp/adadmdat_sql.spool

SQL*Plus: Release 8.0.6.0.0 - Production on Fri Jan 8 14:12:12 2016

(c) Copyright 1999 Oracle Corporation.  All rights reserved.

Connected.

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Checking configuration...

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

PL/SQL procedure successfully completed.

PL/SQL procedure successfully completed.

AOL Schema Name         : APPLSYS

Applications System Name: prod

Source APPLTOP Name     : erpapp1

Target APPLTOP Name     : erpapp1f

ACTIVE_FLAG present?    : TRUE

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Current data

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

APPL_TOP Names in AD_APPL_TOPS:

*PRESEEDED*

GLOBAL

ap302app

testapp5

erpapp1

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Updating tables...

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

PL/SQL procedure successfully completed.

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Updated data

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

APPL_TOP Names in AD_APPL_TOPS:

*PRESEEDED*

GLOBAL

ap302app

testapp5

erpapp1

erpapp1f

6 rows selected.

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Done

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Commit complete.

Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production

With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,

Data Mining and Real Application Testing options

Completed snapshot configuration on target node.

• Follow document Doc ID 217368.1 section 2.4.2

• Change the following values on XML files of both nodes.

s_leastloadedhost=%LeastLoadedHost% 

s_methost=erpapp1

• Change the following values on second(new node) XML file:

s_isForms=enabled

isFormsDev=enabled 

s_metcstatus=enabled

s_formsstatus=enab

• Run autoconfig on second node.

• Run autoconfig on first node.

• Open first node then second node.

Useful apps DBA scripts

• Who is connected, when

sql> select user_name,to_char(last_logon_date,’DD-MON-YYYY HH24:MI:SS’)
from apps.fnd_user
where to_char(last_logon_date,’DD-MON-YYYY’)=to_char(sysdate,’DD-MON-YYYY’);

• Which resposibility is used

sql> select distinct fu.user_name User_Name,fr.RESPONSIBILITY_KEY Responsibility from fnd_user fu,
fnd_responsibility fr, icx_sessions ic
where fu.user_id = ic.user_id AND
fr.responsibility_id = ic.responsibility_id AND
ic.disabled_flag=’N’ AND
ic.responsibility_id is not null AND
ic.last_connect like sysdate;

• Number of user connected to EBS

sql> SELECT COUNT (DISTINCT d.user_name)
FROM apps.fnd_logins a, v$session b, v$process c, apps.fnd_user d
WHERE     b.paddr = c.addr
AND a.pid = c.pid
AND a.spid = b.process
AND d.user_id = a.user_id
AND (d.user_name = ‘USER_NAME’ OR 1 = 1)

• EBS gather stats

sql> execute fnd_stats.gather_table_stats(‘AR’,'RA_CUSTOMER_TRX_ALL’);

• CUSTOM.pll compile: (copy CUSTOM pll under $AU_TOP/resource and compile)

f60gen CUSTOM.pll apps/apps_password module_type=LIBRARY compile_all=YES

• How to change apps password on 11i

• Shutdown all apps services on all nodes.

• Backup these tables                             

                         sql>    create table xxx_fnd_user as select * from FND_USER

                         sql>    create table xxx_FND_ORACLE_USERID asselect * from FND_ORACLE_USERID

• Connect as apps user on sqlplus and run:

                          $ FNDCPASS apps/apps_old 0 Y system/password SYSTEM APPLSYS apps_new

• Edit passwords in these files on all nodes:

                         $IAS_ORACLE_HOME/Apache/modplsql/cfg/wdbsvr.app

                        $ORACLE_HOME/reports60/server/CGIcmd.dat

• How to change module passwords on 11i

$ FNDCPASS apps/apps_password 0 Y system/system_password ORACLE AR new_password

Useful Unix-Linux commands for apps DBA’s

• Search a word on a text(s)

find . -type f -name “*.txt” -exec grep XXWW {} \; -print

• Number of files in a directory:

ls -l | wc -l

• Kill all processes of a user (be careful !)

kill `ps -fu username | awk ‘NR != 1 {print $2}’`

• Find a file under current directory and subdirectories

find . -name “*.trc”

• Find files older than XX days (60 days)

find . -name “*.trc” -mtime +60

• Find files older than XX days and move them to another directory.

find . -name “*.trc” -mtime +60 -exec mv {} /data1/test \

• Find files older than XX days and delete them all (be careful)

find . -name “*.trc” -mtime +60 -exec rm {} \;

• Runing Apache services

ps -ef|grep apache

• Running Concurrent processes

ps -ef|grep FNDLIBR*|grep user

• Running forms services

ps -ef|grep f60*|grep user