Server Utilities :: SQL Loader Handling Different Date Formats In Same CSV File?
Jul 11, 2011
I have another interesting SQL Loader issue. I have created 2 prior topics RE: "Trapping SQL Loader summary counts" and "Using Sql Loader issue". This new issue is along the same lines, but unfortunately the same input csv file has a date field that is has a different input format..
The 10G Oracle Table looks like...
---------------------------
CREATE TABLE PTLIVE.MODULE_CSV_LOADS
( MODULE_ID NUMBER (20),
MODULE_TAGNUMBER VARCHAR2(100),
MODULE_SERIAL_NUMBER NUMBER (20,0),
[code]...
My initial control file looks like...
-----------------
OPTIONS (SKIP=1)
load data
infile 'J:GrowerRound_ModulesCrop2011ProcessedBatch_2011Jul12_081326_746563.csv'
BADFILE 'J:GrowerRound_ModulesCrop2011LogsBatch_2011Jul12_081326_746563.bad'
[code]...
The input csv data file (for this example) contains 3 records. The first is a heading record that is skipped. The 2nd record
is correct and the field entitled "GMT Date" contains a value of
"16/05/2011". The 3rd record, however, has a date of "2011/5/18", which get rejected by the above control file, as the output SQL Loader log indicates...
Record 2: Rejected - Error on table PTLIVE.MODULE_CSV_LOADS, column DATEPICKED.
ORA-01861: literal does not match format string
Now we will be receiving literally hundreds of these csv files and in testing I have found that when I open the csv file (the default is Excel), and Excel must alter its display, as all date appear 100% okay. However, upon opening the csv file with Wordpad, I discovered the dates in the same file had these 2 different formats. So the control file was attempting to concat the input csv file "GMT Date" in one format with the input "GMT_TIME" to load the output value into the Oracle table column
"DATEPICKED" ... but different date formats cause some records to be rejected.
It would be super if SQl Loader could use a IF clause or an OR clause to execute loading the date in one or more input formats. We only expect the 2 foramts DD/MM/YYYY or YYYY/MM/DD....however, there could be 6 different combinations in theory.
I thought about writing a Function or Procedure to analyst the input date and output a standard one to load into the Oracle column
the control file code in this path (c:externalctrl.ctl)
load data infile 'C:externalmy_data.txt' into table emp2 fields terminated by ',' (empno, ename, hiredate, etime, ejob, deptno)
this is the error :
C:>sqlldr scott/tiger control=C:externalctrl.ctl
SQL*Loader: Release 10.2.0.1.0 - Production on Mon May 31 09:45:10 2010 Copyright (c) 1982, 2005, Oracle. All rights reserved. Commit point reached - logical record count 5 C:>
i get flat file and i have set up a control M job so that it runs at a particular time.
initially my control file was as below:
LOAD DATA INFILE 'DFILEabcd.dat' BADFILE 'BDabcd.bad' REPLACE INTO TABLE abcd_table (A position(01:09) CHAR, B position(11:12) CHAR, C position(14:33) CHAR, D position(37:50) char)
this was working fine. control M did not send FAIL message.but later i had to change the ctl file due to requirement. i had to add a when clause.
my code after modification is:
INFILE 'DFILEabcd.dat' BADFILE 'BDabcd.bad' REPLACE INTO TABLE abcd_table when A<>'10' (A position(01:09) CHAR, B position(11:12) CHAR, C position(14:33) CHAR, D position(37:50)
now the control M is sending an erro message after it runs the job. error is Return 5. thats all it gives.
i think it is due to errorlevel 1. in log file it says zero records inserted due to data error. then what is causing control M to send fail message??
sqlloader is loading all the required records correctly.
I have 15 million of records as csv, want to load through sqlloader Is sqlloader is the right option to load high volume of data? I have loaded with 2.5 lac records which has taken 4 mins to load.
I want to load a delimited file that contains many records which contained within the table where I'm going to load a date type field and I need to do this by concatenating three fields
field1 = 1 - this is the day field2 = 11 - this corresponds to the month field3 = 5 - this corresponds to the year
I need is in the field Save as type date 01/11/2005 i don´t know how to do it but I tried as follows but I get error loading.
I'm having a problem with SQL loader and the control file. I want to load a delimited file. The script will eventually be automated where the file name is passed in to the script, it's not a static name.
It's a simple SQL loader Unix script that I have created as follows
Next week I will be getting an input file which will contain over 1000 data columns to be loaded into ORACLE. It's about 6,400 characters in length.
My question is...has ever created a huge ctl file like this to be used for SQLLoader, using so many columns? I will be sending certain columns(data) to certain tables, so it's not just going into 1. It will be about 6 tables.
I am recieving errors when trying to load the control file. The errors are as follows:
SQL*Loader-500 Unable to open file (homework.ctl) SQL*Loader-553 file not found SQL*Loader-559 SYstem error: The system cannot find the file specified.
My control file is located directly in the C drive (C:homework.ctl). The control file contains the following
LOAD DATA INFILE 'c:country.dat' APPEND INTO TABLE homework fields terminated by ',' optionally encloded by '"' (country, month, day) WHEN (month='April')
The command I am entering is:
sqlldr system/password control=homework.ctl
I've tried c:homework.ctl, 'c:homework.ctl', and placing the file in the BIN folder of Oracle.
I am having query regarding sql loader. my data file is comm(,) seperated and I want to load the whole file in oracle table 'bill_temp' except 1st column data of data file.
e.g. File name: bill_file.dat fields seperated by comma ',' values are like emp_id,emp_name,emp_sal,join_date oracle table bill_temp having the below column: emp_name,emp_sal,join_date
Here I want load the emp_name,emp_sal and join_date into oracle table bill_temp. emp_id should not get loaded into table.
Is there any way to skip the loading of particular column data from data file into table?
My requirment is to find out the period names and transactions which are in valid date formats and are less than sysdate and the non date formats are adjustments made by different users for their transactions
The problem here is , When you try to import to a table which has same columns . I skipped the first line when loading . The issue here is the second field is getting split in to the two columns . for e.g. :- Default goes to Child and Remaining goes to the Alias.
infarct there is a tab at the end of the each line. How to set the Sql loader settings correctly so that I can populated the end column in CHILD column only.!!!!
I have been allocated a new PC. Where is the date format used by forms (6i) set up. I previously used to enter 20110519 in the application and it used to accept. It now does not accept the above format.
The nls_date_format on the database is setup as YYYYMMDD.
I want to know how to add date/time in export dump file in Linux using parfile script. I keep getting an error "contain an invalid substitution variables"
my parfile is:
Dumpfile = Daily_Full_%U_`date "+%Y%m%d%H%S"`.dmp or Dumpfile = Daily_Full_%U_`%date%`.dmp
I have an APEX Page , page no1 for person's details that got fields like name, Date of Birth. When user enters the details, it get saved. Now when we want to edit a person's details, we click on that person then it goes to next page, where we can change/details of the person.But when i tried to do that, i am getting Not a valid month error.. log message shows that it is coming from : Automatic Row Processing (DML)- an after SUBMIT process. Is there a way to avoid this error?I changed date formats for several different ways.
I want to load data from a file using sqlldr.I have a table commissions ( technician_id char(5) , tech_name char(30) , Comm_rcd_date DATE , Comm_Paid_date DATE , comm_amt number(10,2) )
my file is 00001,TIMOTHY TROENDLY,2011-03-04T01:45:12+0006,2011-03-04T01:45:12+0007,123.56 00002,KENNETH KLEMENZ,2011-03-04T01:45:12+0006,2011-03-04T01:45:12+0009,123.56 00003,SHUNDAR ARDERY,2011-03-04T01:45:12+0006,2011-03-04T01:45:12+0005,123.56 write a ctl file to load this data.
I installed oracle 10G complete so I can have everything. But now I cannot run sql loader. I check my oracle devsuitehome directory and I cannot find sqlldr.exe
I need to install sql loader separately? I can't find sql loader installer on web.
- 1 - M or P: indicates which table to insert: MASTER or PARENT; - 2 - M or A or B: indicates MASTER, PARENT_A, PARENT_B; - 3:18 - DATA.
Based on the values above, what I need to do is:
1. Load a line to MASTER_TABLE; 2. Load a line to PARENT_TABLE_A pointing to its relative line in MASTER_TABLE; 3. Load a line to PARENT_TABLE_B pointing to its relative line in MASTER_TABLE; 4. In the original file line, there is nothing I can use to join a MASTER line with a PARENT line.
The result would be: MASTER_ID PARENT_DATA 1 PARENT_TABLE_A1 1 PARENT_TABLE_B1 2 PARENT_TABLE_A2 2 PARENT_TABLE_B2
I tried to use both: SEQUENCE and Sequence.NextVall (CurrVal) but they only work when using ROWS=1 and the file I need to load has millions of rows, so I need direct path loading.Also, I read about External Table, but it does not suit my needs because the Application server is not the same as Database server, which is needed by external tables.
in this case is better load the data to a temporary table and then insert to the other tables, I found almost the same question in the topic pointed by the link below: URL....
I want to load geometry into a table using sql*loader. My datafile contains geometry defined as WKT URL...., a standard for geometry and also Oracle has a function called sdo_util.from_wktgeometry.If I'm using a separate 'insert into' statement using this function in sql*plus, there's no problem. But if I'm using the same function in my control-file for sql*loader import, I get a sql*loader-418 error: "bad datafile for column geometrie".
My data file contains records with RECORD_TYPE '01', '02', '03' and '04' in position (30:31) I use sql loader to load this data into a table. I need to load ONLY rows with RECORD_TYPE ='04'. I have to output all the other rows into a file, so I decided to use DISCARD file. Now, those with RECORD_TYPE ='4' have to be loaded into different columns depending on the value in position (267:268).
So, my ctl file should look something like:
WHEN (30:31) = '04' into table MYDATA WHEN (267:268) != 'O ' into table MYDATA WHEN (267:268) = 'O '
and whatever is not '04' goes to discard file.
I tried to use
into table MYDATA WHEN (30:31) = '04' and (267:268) != 'O ' into table MYDATA WHEN (30:31) = '04' and (267:268) = 'O '
but I don't get the right result in terms of the discard file.
Is there any way to put together all these conditions?
The SQL loader somehow is loading only first record. The data file is a csv and the end of line character is a new line. Some text fields have multiple new lines.
Here is my control file
load data infile '/home/devo/c0397105/RuleImport/testLoad/dummyLoad.csv' Truncate into table DUMMY_LOAD_TABLE fields terminated by "," optionally enclosed by '"' ( ID "to_number(:ID)",REQUESTED_GROUP,PURPOSE,COMMENTS) [code]........
We don't have Retail resource type as Dependent System now; the rule will be changed later when Dependent Systems can accept resource types other than application"
