Server Utilities :: Concatenate A Date With SQL Loader From Three Fields
Jul 26, 2010
I want to load a delimited file that contains many records which contained within the table where I'm going to load a date type field and I need to do this by concatenating three fields
field1 = 1 - this is the day
field2 = 11 - this corresponds to the month
field3 = 5 - this corresponds to the year
I need is in the field Save as type date 01/11/2005 i don´t know how to do it but I tried as follows but I get error loading.
I am having a similar problem like above ONLY in UNIX box where my datafile is delimited by "|". The last field is ITM_CMNT declared as VARCHAR2(60) in Oracle. When I have exactly 60bytes in the last field it rejects the record saying actual 61 and max allowed is 60. If i reduce it to < 60bytes then it is stored as a value enclosed with double quotes. The enclosing double quote is on the next line.
"PROC,RAM,FLPY,HD,ACT MTX CLR DSP,D/PCMCIA,TRKBAL,LIT ION BA"
Expected: the one below is exactly 60bytes.
PROC,RAM,FLPY,HD,ACT MTX CLR DSP,D/PCMCIA,TRKBAL,LIT ION BAT LOAD DATA INFILE * INTO TABLE TMPTLI_LAWSON_ITM_MST TRUNCATE FIELDS TERMINATED BY "|" (ITM_NO, HAZ_MAT_CD, ITM_SHRT_DS, ITM_SON "TRIM(:ITM_SON)", ADDED_DT DATE "YYYY-MM-DD", AVL_CD , ITM_CST_AMT, ITM_SLL_AMT, EXCHG_PRC_AMT , ITM_UOM "TRIM(:ITM_UOM)", PCK_QTY INTEGER EXTERNAL, SPC_HNDL_CD "TRIM(:SPC_HNDL_CD)", EFF_DT DATE "YYYY-MM-DD", ITM_CMNT "TRIM(:ITM_CMNT)")
I have 15 million of records as csv, want to load through sqlloader Is sqlloader is the right option to load high volume of data? I have loaded with 2.5 lac records which has taken 4 mins to load.
I have another interesting SQL Loader issue. I have created 2 prior topics RE: "Trapping SQL Loader summary counts" and "Using Sql Loader issue". This new issue is along the same lines, but unfortunately the same input csv file has a date field that is has a different input format..
The 10G Oracle Table looks like... --------------------------- CREATE TABLE PTLIVE.MODULE_CSV_LOADS ( MODULE_ID NUMBER (20), MODULE_TAGNUMBER VARCHAR2(100), MODULE_SERIAL_NUMBER NUMBER (20,0),
[code]...
My initial control file looks like... ----------------- OPTIONS (SKIP=1) load data infile 'J:GrowerRound_ModulesCrop2011ProcessedBatch_2011Jul12_081326_746563.csv' BADFILE 'J:GrowerRound_ModulesCrop2011LogsBatch_2011Jul12_081326_746563.bad'
[code]...
The input csv data file (for this example) contains 3 records. The first is a heading record that is skipped. The 2nd record is correct and the field entitled "GMT Date" contains a value of "16/05/2011". The 3rd record, however, has a date of "2011/5/18", which get rejected by the above control file, as the output SQL Loader log indicates...
Record 2: Rejected - Error on table PTLIVE.MODULE_CSV_LOADS, column DATEPICKED. ORA-01861: literal does not match format string
Now we will be receiving literally hundreds of these csv files and in testing I have found that when I open the csv file (the default is Excel), and Excel must alter its display, as all date appear 100% okay. However, upon opening the csv file with Wordpad, I discovered the dates in the same file had these 2 different formats. So the control file was attempting to concat the input csv file "GMT Date" in one format with the input "GMT_TIME" to load the output value into the Oracle table column "DATEPICKED" ... but different date formats cause some records to be rejected.
It would be super if SQl Loader could use a IF clause or an OR clause to execute loading the date in one or more input formats. We only expect the 2 foramts DD/MM/YYYY or YYYY/MM/DD....however, there could be 6 different combinations in theory.
I thought about writing a Function or Procedure to analyst the input date and output a standard one to load into the Oracle column
the control file code in this path (c:externalctrl.ctl)
load data infile 'C:externalmy_data.txt' into table emp2 fields terminated by ',' (empno, ename, hiredate, etime, ejob, deptno)
this is the error :
C:>sqlldr scott/tiger control=C:externalctrl.ctl
SQL*Loader: Release 10.2.0.1.0 - Production on Mon May 31 09:45:10 2010 Copyright (c) 1982, 2005, Oracle. All rights reserved. Commit point reached - logical record count 5 C:>
I am loading data using sqlldr command in UNIX to an oracle table and want to concatenate timestamp to a file name in the "create_file_name" column in the code below.
I have the below code within the control file..
LOAD DATA TRUNCATE INTO TABLE TABLEA TRAILING NULLCOLS ( file_type POSITION(1:5) CHAR, business_date POSITION(16:23) DATE "YYYYMMDD", create_file_name "FILE_NAME" EXPRESSION "SELECT TO_CHAR(CURRENT_TIMESTAMP(3), 'YYYYMMDDHH24MISS') FROM DUAL")
The load fails with SQL Loader error: "Expecting valid column specification, ",", ")", found keyword EXPRESSION found instead of column. How the timestamp to a filename can be appended?
i load data to this table from a file using sqlldr.
what is the proper data type should i use in control-file for both the fields.? i dont mention any datatye in ctl file which is working fine with given dataset.
using SQLLDR: Looking for a control file solution to move past or bypass extra data fields which are not on destination table. Basically if you have 8 tab delimited fields(terminated by ' ') on a data record; but only need to load 5 of the values from the delimited record; is there a way to ignore/bypass the not needed data. Obviously, the answer would be to massage the data at the OS and removed the 3 unnecessary fields.
However my hands are tied by volume,time, and compliancy. I am familiar with using 'FILLER' for the reverse scenario; but not where you have more data available on the record then exists on the table.
I want to load data from a file using sqlldr.I have a table commissions ( technician_id char(5) , tech_name char(30) , Comm_rcd_date DATE , Comm_Paid_date DATE , comm_amt number(10,2) )
my file is 00001,TIMOTHY TROENDLY,2011-03-04T01:45:12+0006,2011-03-04T01:45:12+0007,123.56 00002,KENNETH KLEMENZ,2011-03-04T01:45:12+0006,2011-03-04T01:45:12+0009,123.56 00003,SHUNDAR ARDERY,2011-03-04T01:45:12+0006,2011-03-04T01:45:12+0005,123.56 write a ctl file to load this data.
I would like to inquire how do I code my select statement if my user enter a search date range(search startDt: 01/08/2012 and search endDt :30/09/2012) and I will like to retrieve only the 7records out from my table as shown below ?
I installed oracle 10G complete so I can have everything. But now I cannot run sql loader. I check my oracle devsuitehome directory and I cannot find sqlldr.exe
I need to install sql loader separately? I can't find sql loader installer on web.
- 1 - M or P: indicates which table to insert: MASTER or PARENT; - 2 - M or A or B: indicates MASTER, PARENT_A, PARENT_B; - 3:18 - DATA.
Based on the values above, what I need to do is:
1. Load a line to MASTER_TABLE; 2. Load a line to PARENT_TABLE_A pointing to its relative line in MASTER_TABLE; 3. Load a line to PARENT_TABLE_B pointing to its relative line in MASTER_TABLE; 4. In the original file line, there is nothing I can use to join a MASTER line with a PARENT line.
The result would be: MASTER_ID PARENT_DATA 1 PARENT_TABLE_A1 1 PARENT_TABLE_B1 2 PARENT_TABLE_A2 2 PARENT_TABLE_B2
I tried to use both: SEQUENCE and Sequence.NextVall (CurrVal) but they only work when using ROWS=1 and the file I need to load has millions of rows, so I need direct path loading.Also, I read about External Table, but it does not suit my needs because the Application server is not the same as Database server, which is needed by external tables.
in this case is better load the data to a temporary table and then insert to the other tables, I found almost the same question in the topic pointed by the link below: URL....
I want to load geometry into a table using sql*loader. My datafile contains geometry defined as WKT URL...., a standard for geometry and also Oracle has a function called sdo_util.from_wktgeometry.If I'm using a separate 'insert into' statement using this function in sql*plus, there's no problem. But if I'm using the same function in my control-file for sql*loader import, I get a sql*loader-418 error: "bad datafile for column geometrie".
My data file contains records with RECORD_TYPE '01', '02', '03' and '04' in position (30:31) I use sql loader to load this data into a table. I need to load ONLY rows with RECORD_TYPE ='04'. I have to output all the other rows into a file, so I decided to use DISCARD file. Now, those with RECORD_TYPE ='4' have to be loaded into different columns depending on the value in position (267:268).
So, my ctl file should look something like:
WHEN (30:31) = '04' into table MYDATA WHEN (267:268) != 'O ' into table MYDATA WHEN (267:268) = 'O '
and whatever is not '04' goes to discard file.
I tried to use
into table MYDATA WHEN (30:31) = '04' and (267:268) != 'O ' into table MYDATA WHEN (30:31) = '04' and (267:268) = 'O '
but I don't get the right result in terms of the discard file.
Is there any way to put together all these conditions?
The SQL loader somehow is loading only first record. The data file is a csv and the end of line character is a new line. Some text fields have multiple new lines.
Here is my control file
load data infile '/home/devo/c0397105/RuleImport/testLoad/dummyLoad.csv' Truncate into table DUMMY_LOAD_TABLE fields terminated by "," optionally enclosed by '"' ( ID "to_number(:ID)",REQUESTED_GROUP,PURPOSE,COMMENTS) [code]........
We don't have Retail resource type as Dependent System now; the rule will be changed later when Dependent Systems can accept resource types other than application"
I am using sqlloader for loading the data into database by using csv file.My csv file is delimited by comma in that i am having a column which is having the , and line feeds targeted to load into a long data type.for example as below
descri,dfdfdfd,dfdfdf, sdfsdf, dfsdfd,
i want to move this column data into a single table column.But due to because of delimited "," it is splitting into number of columns
I had a requirement of loading flatfile into staging table using SQL Loader, One of the columns in the the Flat file is having values FALSE or TRUE and my requirement is that I load 0 for FALSE and 1 for TRUE which can be achieved by simple DECODE function...I did use decode and tried to load several times but did not work.
INFILE 'sql_4ODS.txt' BADFILE 'SQL_4ODS.badtxt' APPEND INTO TABLE members FIELDS TERMINATED BY "|"
[code]...
I did try putting a trim as well as SUBSTR but did not work....the cloumn just doent get any values in the output (just null or say free space)
I have a small problem when I am trying to load data into a table using SQL Loader. The data I am trying to load should be a number, but it is in the format '999,999,999 USD'. When I try to load the data, I am getting an invalid number error, due to the USD (I have already accounted for the thousands seperators). My question is, how can I load the data as a number with USD in the format?
LOAD DATA APPEND INTO TABLE IPGITLREDATA WHEN ITL_REC_TYPE = 'D' FIELDS TERMINATED BY ',' TRAILING NULLCOLS (
[code].....
The data file might have a value of " D " instead of "D" for ITL_REC_TYPE and ITL_REC_TYPE is in the WHEN clause. How can I check for the trimmed value of ITL_REC_TYPE in the WHEN clause ?
I want to populate totale number of record in the file. Usually i get 10000 records per file and i load them using sql loader.I want to also insert the number of records in file while loading the data in table.
How can i achive it.
structure of control file is
load data BADFILE '/backup/temp/rajesh/RIO/BadFiles/FILENAME' append into table ERS_RIO_SRC TRAILING NULLCOLS ( INSTALLATION_ID CHAR
I have loaded 14324590 rows into target tables using sql*loader.I used below consideration during the load process.
1) direct=true,parllel=true 2) unrecoverable 3) disable all indexes and triggers.
But, sql loader takes 21 minutes to load 14324590 rows in database? tuning sql loader process? we cannot change data file because it has given by client.
from Control file, data is getting popupalated in TEST_USER and TEST_TITLE similarly if remove
INTO TABLE TEST_TITLE( TX_SID POSITION(1:3) CHAR, ID_TITLE "get_title_id(:TITLE_NAME)" ,
[code]...
from Control file, TEST_USER and TEST_ROLE is getting populated.
Here RD_ROLE_MASTER script
CREATE TABLE RD_ROLE_MASTER ( "ID_ROLE" NUMBER(38,0) NOT NULL ENABLE, "TX_ROLE_NAME" VARCHAR2(20 BYTE) NOT NULL ENABLE
[code]...
Here is RD_TITLE_MASTER script CREATE TABLE RD_TITLE_MASTER( "ID_TITLE" NUMBER(38,0) NOT NULL ENABLE, "TX_TITLE_NAME" VARCHAR2(25 BYTE) NOT NULL ENABLE);
Insert into RD_TITLE_MASTER (ID_TITLE,TX_TITLE_NAME) values (7,'RED_LOB_ESCALATION_L1'); what is the problem?