Server Administration :: Large Trace Files Generated In Udump?
Oct 24, 2011
We have Oracle 10g(10.2.0.4) RAC on AIX 5.3, there are 3 RAC instances on each node.From Oct 9th, we found one of the instance in node 1 generated a large size trace files in udump. The largest size of the trace file take up 3G. But there's only about 15G $ORACLE_BASE direcotry. After some time, we should delete some trace files for releasing the space.
Here is a part of alert log when this issue happen:
Sun Oct 9 23:18:15 2011
Errors in file /oracle/app/oracle/admin/bzywk/udump/bzywk1_ora_3166258.trc:
ORA-00600: internal error code, arguments: [17087], [0x70000010DF9F580], [], [], [], [], [], []
Sun Oct 9 23:18:16 2011
Trace dumping is performing id=[cdmp_20111009231816]
[code]....
I checked with some trace files, I found all these files contains a huge info of processes.
We are facing one issue on one of the database. The database is generating large trace files(14000) from last two days. That consumes around 15G space on the disk. And the content of the trace files is not having any meaningful message to debug:
*** TRACE DUMP CONTINUED FROM FILE /apps/oracle/admin/fs90uat/bdump/fs90uat_p050_23966.trc ***
... (Many lines with above message)
The alert log is having one repeated error yesterday:
Thu May 6 22:00:03 2010 Errors in file /apps/oracle/admin/fs90uat/bdump/fs90uat_j000_11811.trc: ORA-12012: error on auto execute of job 2647927 ORA-04063: ORA-04063: package body "ORACLE_OCM.MGMT_DB_LL_METRICS" has errors ORA-06508: PL/SQL: could not find program unit being called: "ORACLE_OCM.MGMT_DB_LL_METRICS" ORA-06512: at line 1
The corresponding trace file is having error:
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production With the Partitioning, OLAP, Data Mining and Real Application Testing options ORACLE_HOME = /apps/oracle/product/10.2.0/db_1 System name: SunOS Node name: corpqadb30 [Code] .......
I am facing problem in user_dump_dest directory...I have noticed that there are a lot of trace files with huge size in MBs.I clean it and after 4 days there are 40G of size..
I have enabled the below trace event before running query on my session.
ALTER SESSION SET events = '19027 trace name context forever, level 0x1000';ALTER SESSION SET tracefile_identifier = store_trace;
Then I ran my query, and upon the finishing of the query, I ran the below to disable trace.
ALTER SESSION SET events = '19027 trace name context off, level 0x1000';
Now I checked back on the trace folder in my Oracle DB directory structure, but cannot find any file created by using the above set tracefile_identifier. I had previously run the query with "sql_trace" ON with tracefile_identifier, and successfully got the tracefile generated by using the identifier.
I am considering all of the capabilities and benefits of using Data Pump for exporting and importing extremely large data files. Would like to know if importing to tape is possible? If so, would the data be accessible if needed later?
The trace directory is full of trc files with the text below:
--------Dumping Sorted Master Trigger List -------- Trigger Owner : INPUT Trigger Name : CUS_TST --------Dumping Trigger Sublists --------
There is like a file generated every minute, and i cant stop it from happening.I have tried setting the trace_enabled parameter to FALSE but no success.
It always used to be that Automatic Memory Management and Linux huge pages were incompatible: you had to use one or the other. But 11.2.0.2 has new parameter, USE_LARGE_PAGES. This isn't documented apart from a few articles on metalink, but Googling it suggests that if it is on TRUE (the default) or ONLY then I can use AMM with huge pages.
I have a problem some user modified the objects and now objects became invalid .How to trace the ip address and operating system username and service ip who modified the objects?
un-documented parameter _trace_files_public / I want to set this to true so my app team can review trace files. better way to proceed to open read permissions for non oracle users.
I am using Oracle RAC 11gR2.I am planing to move my old trace files to an archive location.
I have the below questions concerning the users and groups used by Oracle RAC when moving the trace files.It is related to the permissions but it is about oracle behavior.
The trace files of database (oracle user) are created as below:
284 -rw-r----- 1 oracle asmadmin 283571 Aug 20 18:08 PPRD1_mmon_13893716.trc 4 -rw-r----- 1 oracle asmadmin 125 Aug 20 18:10 PPRD1_gcr0_11730998.trm 4 -rw-r----- 1 oracle asmadmin 2833 Aug 20 18:10 PPRD1_gcr0_11730998.trc 580 -rw-r----- 1 oracle asmadmin 588489 Aug 20 18:12 PPRD1_lmhb_15859792.trm
[code]....
when moving the files using the oracle user it will be created as oracle:oinstall and not oracle:asmadmin since asmadmin is not a group of the oracle user.
1. How oracle created my trace and log files with permission oracle:asmadmin?
2. How should I move these files? I moved it using the Oracle user and it was created as oracle:oinstall with Operating system error "unable to duplicate owner and mode after move"
Some trace files were deleted with rm command. Obviously they were active and now the free space does not free up (over 9GB). How can this be fixed without database interruption?
I have a server with Red Hat EL 5.5 running an oracle database 10g R2 and an Oracle Agent 10.2.0.5The disk ran out of space and I realized that there are a lot of files stating with core.xxx such as :
-rw------- 1 gridagent oinstall 17M Aug 2 13:10 core.10348 -rw------- 1 gridagent oinstall 17M Aug 2 13:15 core.10827 -rw------- 1 gridagent oinstall 17M Aug 2 12:30 core.4129 -rw------- 1 gridagent oinstall 17M Aug 2 12:35 core.4772
What's that ?Why are those files generated at $AGENT_HOME/<HOST>_<SID>/sysman/log/ ?
There are more than 24 GB of those files
in nmc.log I can see lines such as: NMC-00000 2010-08-02 13:25:39 [11760, nmcdbg.c,0440]TRC: Debug context enabled. NMC-20020 2010-07-30 01:31:58 [23083, nmccole.c,0725]ERR: Could not collect using DSGA Collection method NMC-20014 2010-07-29 01:31:56 [11258, nmccole.c,0822]ERR: Could not attach to the SGA
is it possible to upload very large files in oracle's tables. For example 1-2 gigabyte video file or even more. In other words is it possible to use oracle as file server to upload very large files and store them?
The steps to move OMF files in ASM. I tried the following and was not successful.
RMAN> switch database to copy;
datafile 1 switched to datafile copy "+DATA01/pa01pod_im1l059p/datafile/system.357.809972853" datafile 2 switched to datafile copy "+DATA01/pa01pod_im1l059p/datafile/sysaux.363.809972837" datafile 3 switched to datafile copy "+DATA01/pa01pod_im1l059p/datafile/undotbs1.365.809972737" datafile 4 switched to datafile copy "+DATA01/pa01pod_im1l059p/datafile/users.361.809972859" datafile 5 switched to datafile copy "+DATA01/pa01pod_im1l059p/datafile/undotbs2.360.809972761" datafile 6 switched to datafile copy "+DATA01/pa01pod_im1l059p/datafile/undotbs3.359.809972787" datafile 7 switched to datafile copy "+DATA01/pa01pod_im1l059p/datafile/undotbs4.358.809972811"
RMAN> alter database open resetlogs;
RMAN-00571: =========================================================== RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS =============== RMAN-00571: =========================================================== RMAN-03002: failure of alter db command at 03/13/2013 16:30:05 ORA-01139: RESETLOGS option only valid after an incomplete database recovery
RMAN> alter database open;
so switch worked reset logs says can't use it there so I just try to open and it just hangs.
1) Can we fetch 'select statements' from redo log files through the use of log miner utility or any other? (I think redo log file contains only insert,update,delete and DDL/DCL commands only)
2) If "No" to the above answer then how can i fetch all select statements fired on the system for a day or particular time. (setting of sql_trace may be the one of them, but can it be possible for system level)
I have got backup pieces in ASM, I guess about more than 100 files. Now, I need to copy all of them from ASM to FileSystem, there are 2 methods still now:
1- Copy from ASM to FileSystem using cp command. 2- Copy from ASM to FileSystem using DBMS_FILE_TRANSFER.
But:
In the first method, when I copy one file, I took more than 1 minute, so the following script would take me more than 1 days (I guess so).
#!/bin/ksh # # This script copies files from FRA on ASM to local disk # ORACLE_SID=+ASM2 ASMLS=/vasgatedb/app/vsgbkp/asm_ls.txt ##{ASM files list}
[Code]...
The second method, DBMS_FILE_TRANSFER took me less than 1 second to copy one file completely.
sys@VSGDB> set timing on sys@VSGDB> exec dbms_file_transfer.COPY_FILE('asm_dir','level_0_vsgdb_9998_813844797.bkp','fs_dir','level_0_vsgdb_9998_813844797.bkp');
PL/SQL procedure successfully completed.
Elapsed: 00:00:00.38
Of course, I wish I used the second method as soon as possible, however, said I below, I've got about ~200 files, and I could not copy one by one file.
I have got oracle 11g release 1, I need recomendation, how can I delete safely content from the folder C:oraclediag dbmsinstancenameinstancenameincident?
ora-00257 archiver error. connect internal only until freed
when we tried to remove the unwanted arc files thro ASMCMD,we are getting below error:
ASMCMD> rm -ef 2011_04_05/ Unknown option: e usage: rm [-rf] <name1 name2 . . .> ASMCMD> rm -rf 2011_04_05/ ORA-15032: not all alterations performed ORA-15028: ASM file '+XCOM_BACKUP_DG/TXCOM/ARCHIVELOG/2011_04_05/thread_2_seq_27215.1143.747641143' not dropped; currently being accessed (DBD ERROR: OCIStmtExecute) ORA-15032: not all alterations performed ORA-15028: ASM file '+XCOM_BACKUP_DG/TXCOM/ARCHIVELOG/2011_04_05/thread_3_seq_21762.826.747641143' not dropped; currently being accessed (DBD ERROR: OCIStmtExecute) ORA-15032: not all alterations performed ORA-15177: cannot operate on system aliases (DBD ERROR: OCIStmtExecute)
I'm facing problem with archive log file size, Archive logs are generated with only of 90m or 92m or 94m(Variable sizes of less than 100m), Although i had set 100m for each of my redo log file. Here i'm providing my create db script for your reference. I want to know why the log switches before it reaches 100m.Is there any connection of intial 10m for my .dbf files.
I am trying to install Oracle 11g Release 2 on Redhat Enterprise Linux 5.2 (vmware image) but after 50% installation it through errors while copying files on the server. I also tried on Windows 2003 Server R2 (physical m/c as well as vmware image) and faced same problems. I did installation of Oracle 11g Release 1 and didn't find any problems.
I am having 11.2.0.1.0 database with windows2k3 OS. For ORA-609 error i have started server side tracing by adding TRACE_LEVEL_SERVER=16,TRACE_LEVEL_DIRECOTRY=D:TRACE,and DIAG_ADR_ENABLED=off in sqlnet.ora file and i reload the listener.
Now i want to stop tracing so i have tried TRACE_LEVEL_SERVER=off but it wouldnot work. After that i remove TRACE_LEVEL_SERVER=16,TRACE_LEVEL_DIRECOTRY=D:TRACE from sqlnet.ora file and reload the listener but trace file generation is going on D:TRACE. stop server side tracing.
