Server Utilities :: SQL Loader Handling Different Date Formats In Same CSV File?
Jul 11, 2011
I have another interesting SQL Loader issue. I have created 2 prior topics RE: "Trapping SQL Loader summary counts" and "Using Sql Loader issue". This new issue is along the same lines, but unfortunately the same input csv file has a date field that is has a different input format..
The 10G Oracle Table looks like...
---------------------------
CREATE TABLE PTLIVE.MODULE_CSV_LOADS
( MODULE_ID NUMBER (20),
MODULE_TAGNUMBER VARCHAR2(100),
MODULE_SERIAL_NUMBER NUMBER (20,0),
[code]...
My initial control file looks like...
-----------------
OPTIONS (SKIP=1)
load data
infile 'J:GrowerRound_ModulesCrop2011ProcessedBatch_2011Jul12_081326_746563.csv'
BADFILE 'J:GrowerRound_ModulesCrop2011LogsBatch_2011Jul12_081326_746563.bad'
[code]...
The input csv data file (for this example) contains 3 records. The first is a heading record that is skipped. The 2nd record
is correct and the field entitled "GMT Date" contains a value of
"16/05/2011". The 3rd record, however, has a date of "2011/5/18", which get rejected by the above control file, as the output SQL Loader log indicates...
Record 2: Rejected - Error on table PTLIVE.MODULE_CSV_LOADS, column DATEPICKED.
ORA-01861: literal does not match format string
Now we will be receiving literally hundreds of these csv files and in testing I have found that when I open the csv file (the default is Excel), and Excel must alter its display, as all date appear 100% okay. However, upon opening the csv file with Wordpad, I discovered the dates in the same file had these 2 different formats. So the control file was attempting to concat the input csv file "GMT Date" in one format with the input "GMT_TIME" to load the output value into the Oracle table column
"DATEPICKED" ... but different date formats cause some records to be rejected.
It would be super if SQl Loader could use a IF clause or an OR clause to execute loading the date in one or more input formats. We only expect the 2 foramts DD/MM/YYYY or YYYY/MM/DD....however, there could be 6 different combinations in theory.
I thought about writing a Function or Procedure to analyst the input date and output a standard one to load into the Oracle column
View 11 Replies
ADVERTISEMENT
May 31, 2010
what i miss to load date and time from text file to oracle table through sqlloader
this is my data in this path (c:externalmy_data.txt)
7369,SMITH,17-NOV-81,09:14:04,CLERK,20
7499,ALLEN,01-MAY-81,17:06:08,SALESMAN,30
7521,WARD,09-JUN-81,17:06:30,SALESMAN,30
7566,JONES,02-APR-81,09:24:10,MANAGER,20
7654,MARTIN,28-SEP-81,17:24:10,SALESMAN,30
my table in database emp2
create table emp2 (empno number,
ename varchar2(20),
hiredate date,
etime date,
ejob varchar2(20),
deptno number);
the control file code in this path (c:externalctrl.ctl)
load data
infile 'C:externalmy_data.txt'
into table emp2
fields terminated by ','
(empno, ename, hiredate, etime, ejob, deptno)
this is the error :
C:>sqlldr scott/tiger control=C:externalctrl.ctl
SQL*Loader: Release 10.2.0.1.0 - Production on Mon May 31 09:45:10 2010
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Commit point reached - logical record count 5
C:>
View 21 Replies
View Related
Aug 21, 2010
i get flat file and i have set up a control M job so that it runs at a particular time.
initially my control file was as below:
LOAD DATA
INFILE 'DFILEabcd.dat'
BADFILE 'BDabcd.bad'
REPLACE
INTO TABLE abcd_table
(A position(01:09) CHAR,
B position(11:12) CHAR,
C position(14:33) CHAR,
D position(37:50) char)
this was working fine. control M did not send FAIL message.but later i had to change the ctl file due to requirement. i had to add a when clause.
my code after modification is:
INFILE 'DFILEabcd.dat'
BADFILE 'BDabcd.bad'
REPLACE
INTO TABLE abcd_table
when A<>'10'
(A position(01:09) CHAR,
B position(11:12) CHAR,
C position(14:33) CHAR,
D position(37:50)
now the control M is sending an erro message after it runs the job. error is Return 5. thats all it gives.
i think it is due to errorlevel 1. in log file it says zero records inserted due to data error. then what is causing control M to send fail message??
sqlloader is loading all the required records correctly.
View 10 Replies
View Related
Sep 9, 2010
I am attempting to insert date data into a column using sqlldr...Here's the current format:
2010-03-01 00:20:19.277
So far, I haven't gotten anything to work. I would like to trim the .277 from the existing date. Here's my latest attempt:
birthdate DATE "to_date(substr(birthdate,1,19),'YYYY-MM-DD HH24:MI:SS')"
View 6 Replies
View Related
Jun 6, 2013
I have 15 million of records as csv, want to load through sqlloader Is sqlloader is the right option to load high volume of data? I have loaded with 2.5 lac records which has taken 4 mins to load.
View 9 Replies
View Related
Jan 8, 2011
I am trying to load my data from a CSV file to an oracle table using SQL LOADER. What should be the syntax of DATE MASK for the below kind of data.
02MAR1948:00:00:00.000
24SEP1950:00:00:00.000
I am using the following control file
LOAD DATA
INTO TABLE TW_EXTDATA.PATIENT_VISITS_MERGE_39950
REPLACE
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
[code]...
View 2 Replies
View Related
Jul 26, 2010
I want to load a delimited file that contains many records which contained within the table where I'm going to load a date type field and I need to do this by concatenating three fields
field1 = 1 - this is the day
field2 = 11 - this corresponds to the month
field3 = 5 - this corresponds to the year
I need is in the field Save as type date 01/11/2005 i don´t know how to do it but I tried as follows but I get error loading.
"TO_DATE (TO_CHAR (: field1 ||'/'|| DECODE (: field2, 1, 'JAN', 2, 'FEB', 3, 'MAR', 4, 'APR', 5, 'MAY', 6 , 'JUN', 7, 'JUL', 8, 'AUG', 9, 'SEP', 10, 'OCT', 11, 'NOV', 12, 'DEC': field2 )||'/'|| : field3)) "
View 3 Replies
View Related
Sep 5, 2010
1) can we use a CSV file as a Data file in any format (fixed, delimited...) of Sql loader. I tried, but not succeeded.
2) if not then tell me the reason for it....
3) Also tell me is there any restriction on using the file format for a datafile?
View 18 Replies
View Related
Nov 11, 2011
My sql loader is returning a code 2 but not creating a bad file for the rejected records. also logs look good.
View 14 Replies
View Related
Mar 19, 2010
I'm having a problem with SQL loader and the control file. I want to load a delimited file. The script will eventually be automated where the file name is passed in to the script, it's not a static name.
It's a simple SQL loader Unix script that I have created as follows
Unix file called test_load
# Auto Load
#
#
export data_file=/dev/test_$1$2.csv;
export ORACLE_HOME=/u01/oracle/product/10.2.0;
export ORACLE_SID=XXX;
export PATH=$PATH:$ORACLE_HOME/bin;
sqlldr userid=XXX/XXX data=$data_file
control=/dev/cntrl/test.ctl
errors=99999
bad=/dev/bad/test_$1$2_$$.bad
log=/dev/logs/test_$1$2_$$.log
the top of my control file is as follows
load data
truncate
into table test
fields terminated by "|"
when record_type = 'AA'
(
running at the prompt ./test_load myload 20100319
The following error occurs
SQL*Loader-350: Syntax error at line 5.
Expecting "(", found keyword when.
when record_type = 'AA'
^
I believe the format of my control file is correct but for some reason it won't load.
View 10 Replies
View Related
Jan 28, 2011
The reason "why sql loader's CONTROL FILE is not working, if that location has space in between"
E.g. - C:practice for sqlloadercontrol1.ctl
But it works if we remove the space in between the path as below:
E.g. - C:practice_for_sqlloadercontrol1.ctl
View 3 Replies
View Related
Dec 3, 2010
Next week I will be getting an input file which will contain over 1000 data columns to be loaded into ORACLE. It's about 6,400 characters in length.
My question is...has ever created a huge ctl file like this to be used for SQLLoader, using so many columns? I will be sending certain columns(data) to certain tables, so it's not just going into 1. It will be about 6 tables.
View 39 Replies
View Related
Mar 9, 2012
I am recieving errors when trying to load the control file. The errors are as follows:
SQL*Loader-500 Unable to open file (homework.ctl)
SQL*Loader-553 file not found
SQL*Loader-559 SYstem error: The system cannot find the file specified.
My control file is located directly in the C drive (C:homework.ctl). The control file contains the following
LOAD DATA
INFILE 'c:country.dat'
APPEND INTO TABLE homework
fields terminated by ',' optionally encloded by '"'
(country, month, day)
WHEN (month='April')
The command I am entering is:
sqlldr system/password control=homework.ctl
I've tried c:homework.ctl, 'c:homework.ctl', and placing the file in the BIN folder of Oracle.
View 7 Replies
View Related
Oct 26, 2012
I am having query regarding sql loader. my data file is comm(,) seperated and I want to load the whole file in oracle table 'bill_temp' except 1st column data of data file.
e.g.
File name: bill_file.dat
fields seperated by comma ','
values are like
emp_id,emp_name,emp_sal,join_date
oracle table bill_temp having the below column:
emp_name,emp_sal,join_date
Here I want load the emp_name,emp_sal and join_date into oracle table bill_temp.
emp_id should not get loaded into table.
Is there any way to skip the loading of particular column data from data file into table?
View 12 Replies
View Related
Aug 16, 2010
i have a table with the following description
create table gl_periods(period_name varchar2(10),transactions number (2) );
with the data as :
period_name transactions
------------ --------------
JAN-10 12
FEB-10 12
MAR-10 8
APR-10 23
ADJ_TOM-10 25
MAY-10 37
JUN-10 41
JUL-10 10
PHY_JAY-10 6
AUG-10 14
SEP-10 22
My requirment is to find out the period names and transactions which are in valid date formats and are less than sysdate and the non date formats are adjustments made by different users for their transactions
View 8 Replies
View Related
May 8, 2011
I have an Excel spreadsheet that just had a format change. The fourth column is new. Order of columns in Excel is:
oldcol1
oldcol2
oldcol3
oldcol4
newcol
oldcol5
Sqlldr script is in order of the .csv file just listed. Oracle table is in order
oldcol1
oldcol2
oldcol3
oldcol4
oldcol5
newcol
When I run the sqlldr script, the information is loaded:
oldcol1
oldcol2
oldcol3
oldcol4
newcol in oldcol5
so that all the information is loaded incorrectly. Out side of having to change the table, is there anything I can do to make it load correctly?
View 8 Replies
View Related
Jan 4, 2012
I am trying to import data from below content.
Flat File
PARENTCHILDALIAS
PLAN_PCOTDefaultPlanning Customer
1001_BTPCOTDefaultGeneral Planning Customer
2000_BTPCOTDefaultNational Account Planning Customer
3000_BTPCOTDefaultDistributor Planning Customer
3010_BTPCOTDefaultEducation Planning Customer
3020_BTPCOTDefaultResearch Planning Customer
OPT1_PCOTDefaultOption 1 Planning customer
OPT2_PCOTDefaultOption 2 Planning customer
OPT3_PCOTDefaultOption 3 Planning customer
The problem here is , When you try to import to a table which has same columns . I skipped the first line when loading . The issue here is the second field is getting split in to the two columns . for e.g. :- Default goes to Child and Remaining goes to the Alias.
infarct there is a tab at the end of the each line. How to set the Sql loader settings correctly so that I can populated the end column in CHILD column only.!!!!
OPTIONS ( SKIP=1)
LOAD DATA
INFILE 'FlatFile.txt'
BADFILE ''FlatFile.bad'
DISCARDFILE ''FlatFile.dsc'
INTO TABLE "table"
FIELDS TERMINATED BY X'9'
OPTIONALLY ENCLOSED BY "''" TRAILING NULLCOLS
(PARENT,
CHILD,
ALIAS CONSTANT '')
View 5 Replies
View Related
Nov 24, 2011
Load the selected records from the flat file using SQL*Loader.
I have a flat file it's having 100 records, I want to load first 10 records from the file using SQL*LOADER.
View 8 Replies
View Related
May 19, 2011
I have been allocated a new PC. Where is the date format used by forms (6i) set up. I previously used to enter 20110519 in the application and it used to accept. It now does not accept the above format.
The nls_date_format on the database is setup as YYYYMMDD.
View 2 Replies
View Related
Jul 11, 2013
I want to know how to add date/time in export dump file in Linux using parfile script. I keep getting an error "contain an invalid substitution variables"
my parfile is:
Dumpfile = Daily_Full_%U_`date "+%Y%m%d%H%S"`.dmp or
Dumpfile = Daily_Full_%U_`%date%`.dmp
View 1 Replies
View Related
Jul 18, 2013
I have an APEX Page , page no1 for person's details that got fields like name, Date of Birth. When user enters the details, it get saved. Now when we want to edit a person's details, we click on that person then it goes to next page, where we can change/details of the person.But when i tried to do that, i am getting Not a valid month error.. log message shows that it is coming from : Automatic Row Processing (DML)- an after SUBMIT process. Is there a way to avoid this error?I changed date formats for several different ways.
View 1 Replies
View Related
Jan 15, 2013
My Table structure
column1 varchar(10)
column2 Date
Column3 varcahr(2)
Column4 varcahr(2)
By Data file
asds 12/12/2001asas
textsd asds
asds 12/12/2001asas
ramkiy asds
I still want to insert row 2 and row 4 into table by defaulting the date. how can I handle this in control file?
View 14 Replies
View Related
Jun 6, 2012
I am not able to load complete date along with time in the date column. here is my table desc
DESC STAGE
Name Null Type
----------------------------------
TABLE_NAME NOT NULL VARHCAR(20)
RECORDCOUNT NUMBER
CREATED_DATE NOT NULL DATE
my control file is like this
LOAD DATA
APPEND
INTO TABLE SCOOP.STAGE
FIELDS TERMINATED BY ","
( TABLE_NAME
,RECORDCOUNT
,CREATED_DATE DATE(16) "YYYYMMDDHH:Mi:SS"
)
the data gets loaded, but it appears like this in the table
HIGHSCHOOL3080606-JUN-12
MIDDLESCHOOL8768006-JUN-12
BUT I WANT COMPLETE DATE AND TIME (HH:MI:SS) , HOW CAN I GET IT (THIS IS HOW I WANT 06-JUN-12 11:07:33)
View 10 Replies
View Related
Oct 18, 2012
I want to load data from a file using sqlldr.I have a table commissions
(
technician_id char(5)
, tech_name char(30)
, Comm_rcd_date DATE
, Comm_Paid_date DATE
, comm_amt number(10,2)
)
my file is
00001,TIMOTHY TROENDLY,2011-03-04T01:45:12+0006,2011-03-04T01:45:12+0007,123.56
00002,KENNETH KLEMENZ,2011-03-04T01:45:12+0006,2011-03-04T01:45:12+0009,123.56
00003,SHUNDAR ARDERY,2011-03-04T01:45:12+0006,2011-03-04T01:45:12+0005,123.56
write a ctl file to load this data.
View 6 Replies
View Related
Feb 4, 2011
I installed oracle 10G complete so I can have everything. But now I cannot run sql loader. I check my oracle devsuitehome directory and I cannot find sqlldr.exe
I need to install sql loader separately? I can't find sql loader installer on web.
View 3 Replies
View Related
Jul 7, 2010
SQL Loader - How to implement/Best solution.
I have 3 tables with their columns:
- MASTER_TABLE - MASTER_ID, DATA;
- PARENT_TABLE_A - MASTER_ID, DATA;
- PARENT_TABLE_B - MASTER_ID, DATA.
And the file I need to import has lines like the ones below:
MMMASTER_TABLE1
PAPARENT_TABLE_A1
PBPARENT_TABLE_B1
MMMASTER_TABLE2
PAPARENT_TABLE_A2
PBPARENT_TABLE_B2
MMMASTER_TABLE3
PAPARENT_TABLE_A3
PBPARENT_TABLE_B3
The line means:
- 1 - M or P: indicates which table to insert: MASTER or PARENT;
- 2 - M or A or B: indicates MASTER, PARENT_A, PARENT_B;
- 3:18 - DATA.
Based on the values above, what I need to do is:
1. Load a line to MASTER_TABLE;
2. Load a line to PARENT_TABLE_A pointing to its relative line in MASTER_TABLE;
3. Load a line to PARENT_TABLE_B pointing to its relative line in MASTER_TABLE;
4. In the original file line, there is nothing I can use to join a MASTER line with a PARENT line.
The result would be:
MASTER_ID PARENT_DATA
1 PARENT_TABLE_A1
1 PARENT_TABLE_B1
2 PARENT_TABLE_A2
2 PARENT_TABLE_B2
I tried to use both: SEQUENCE and Sequence.NextVall (CurrVal) but they only work when using ROWS=1 and the file I need to load has millions of rows, so I need direct path loading.Also, I read about External Table, but it does not suit my needs because the Application server is not the same as Database server, which is needed by external tables.
in this case is better load the data to a temporary table and then insert to the other tables, I found almost the same question in the topic pointed by the link below: URL....
View 7 Replies
View Related
Mar 10, 2010
I want to load geometry into a table using sql*loader. My datafile contains geometry defined as WKT URL...., a standard for geometry and also Oracle has a function called sdo_util.from_wktgeometry.If I'm using a separate 'insert into' statement using this function in sql*plus, there's no problem. But if I'm using the same function in my control-file for sql*loader import, I get a sql*loader-418 error: "bad datafile for column geometrie".
Why and how I can import WKT using sql*loader?
-- data file
id;geometrie
1;POINT(120123.123 485345.789)
2;LINESTRING(123456.01 482543.21, 125764.76 483444.11)
3;POLYGON((121121.22 484394.22, 122887.444 484721.48, 122911.098 486382.45, 121005.21 486592.01, 121121.22 484394.22),(121922.56 485333.23, 122010.22 485854.83, 121922.56 485333.23))
4;MULTIPOINT((120586 483958.33),(120635 483726.11))
5;MULTILINESTRING((117948 480284, 118215 481236),(118475 481604, 120462 482822))
6;MULTIPOLYGON(((123678 481948, 124654 485215, 123678 481948),(127321 488321, 124907 483921, 127321 488321)))
[code]....
View 15 Replies
View Related
Nov 10, 2011
I want to use SQL loader in a procedure..will it be possible to use it in procedure and if yes then how..?
View 4 Replies
View Related
Sep 25, 2012
My data file contains records with RECORD_TYPE '01', '02', '03' and '04' in position (30:31) I use sql loader to load this data into a table. I need to load ONLY rows with RECORD_TYPE ='04'. I have to output all the other rows into a file, so I decided to use DISCARD file. Now, those with RECORD_TYPE ='4' have to be loaded into different columns depending on the value in position (267:268).
So, my ctl file should look something like:
WHEN (30:31) = '04'
into table MYDATA
WHEN (267:268) != 'O '
into table MYDATA
WHEN (267:268) = 'O '
and whatever is not '04' goes to discard file.
I tried to use
into table MYDATA
WHEN (30:31) = '04' and (267:268) != 'O '
into table MYDATA
WHEN (30:31) = '04' and (267:268) = 'O '
but I don't get the right result in terms of the discard file.
Is there any way to put together all these conditions?
View 3 Replies
View Related
Sep 21, 2011
The SQL loader somehow is loading only first record. The data file is a csv and the end of line character is a new line. Some text fields have multiple new lines.
Here is my control file
load data
infile '/home/devo/c0397105/RuleImport/testLoad/dummyLoad.csv'
Truncate
into table DUMMY_LOAD_TABLE
fields terminated by "," optionally enclosed by '"'
( ID "to_number(:ID)",REQUESTED_GROUP,PURPOSE,COMMENTS)
[code]........
We don't have Retail resource type as Dependent System now; the rule will be changed later when Dependent Systems can accept resource types other than application"
View 1 Replies
View Related
