Troubleshooting Concurrent Request ORA-20100 error

When running "GMFRLAYR module: GMF Recreate Batch / Period Layers" Concurrent request we got the below error:

ORACLE error 20100 in FDPSTP Cause: FDPSTP failed due to ORA-20100: Error: FND_FILE failure. Unable to create file, o1152922.tmp in the directory, /usr/tmp. You will find more information in the request log. ORA-06512: at "APPS.FND_FILE", line 417 

- All configurations seems to be OK:

As the DB owner:

SQL> sho parameter utl_file_dir

NAME                     TYPE     VALUE
------------------------------------ ----------- ------------------------------
utl_file_dir                 string     /usr/tmp, /usr/tmp, /Test/FINT
                         EST/db/tech_st/11.1.0/appsutil
                         /outbound/FINTEST_erpmigrate02
                         , /usr/tmp
SQL> 

As the MT owner:

[apfintest@erpmigrate02 ~]$ echo $APPLPTMP
/usr/tmp
[apfintest@erpmigrate02 ~]$ ls -l /usr/|grep tmp
lrwxrwxrwx   1 root root     10 Nov  4 13:01 tmp -> ../var/tmp
[apfintest@erpmigrate02 ~]$ ls -l /var|grep tmp
drwxrwxrwx  3 root root  4096 Nov 13 11:31 tmp
[apfintest@erpmigrate02 ~]$ 

- We have enough space for new logs:

[apfintest@erpmigrate02 ~]$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda2             254G  5.7G  235G   3% /
/dev/sdb2             943G  348G  547G  39% /Oracle
/dev/sdb1             946G  348G  551G  39% /Test
/dev/sda1             289M   32M  243M  12% /boot
tmpfs                 7.9G   17M  7.9G   1% /dev/shm
[apfintest@erpmigrate02 ~]$

- Trying to use the package used by the application to write the file:

SQL> sho user
USER is "APPS"
SQL> exec FND_FILE.PUT_LINE(FND_FILE.LOG, 'testing /usr/tmp folder');

PL/SQL procedure successfully completed.

SQL> !ls -ltr|tail
-rwxrwxrwx 1 orscm     dba           0 Nov 13 11:40 o1152928.tmp
-rwxrwxrwx 1 orscm     dba           0 Nov 13 11:40 l1152928.tmp
-rwxrwxrwx 1 orscm     dba           0 Nov 13 11:41 o1152959.tmp
-rwxrwxrwx 1 orscm     dba           0 Nov 13 11:41 l1152959.tmp
-rwxrwxrwx 1 orscm     dba           0 Nov 13 11:41 o1152946.tmp
-rwxrwxrwx 1 orscm     dba           0 Nov 13 11:41 l1152946.tmp
-rwxrwxrwx 1 orscm     dba           0 Nov 13 11:41 o1152916.tmp
-rwxrwxrwx 1 orscm     dba           0 Nov 13 11:41 l1152916.tmp
-rw-r--r-- 1 orfintest oinstall      0 Nov 13 11:41 o1154916.tmp
-rw-r--r-- 1 orfintest oinstall     24 Nov 13 11:41 l1154916.tmp

SQL> !cat l1154916.tmp
testing /usr/tmp folder

SQL>

- At last I found that the /usr/tmp folder is shared between two instances! and the same file name got generated by both instances!

- At this point you have two options:

a)

1. Stop all MT services.
2. Change $APPLPTMP value to another directory "ensure it is the first value listed by utl_file_dir DB parameter"

3. Run autoconfig on the MT nodes

4. Restart all services
5. Re-test the failed process

b)

As a quick solution "should not be applied unless you really need an urgent solution"

[root@erpmigrate02 tmp]# chmod 777 /usr/tmp/*
[root@erpmigrate02 tmp]# 

re-run your CR again...

Regards,

Mohamed