SQL Loader - Null Columns When Using Terminated And Optionally Enclosed Delimiters
Sep 25, 2012
I regularly upload various tab-delimited text data files with SQL Loader. The control files always specify TERMINATED BY X'09'. Certain columns in those data files may be null for some rows, i.e. there is no character between two subsequent tabs. It always works like a clock.
Now, I have run into a specific case where I have to strip a data column of double quotes that may or may not enclose the actual data (side effect of a text export from Excel).
I tried simply adding OPTIONALLY ENCLOSED BY '"' behind the terminated delimiter and it did work for the file and column in question. Still, with this new option, files with null values are no longer decoded correctly. The loader seems to simply skip those values, which provokes column shifts and results in either a corrupt load, or a load failure.
For now, I've got a workaround dropping the new option and executing an SQL script eliminating double quotes directly in the database, but that obviously cannot last.
I have 8 columns. Some of them might be null.I want to display all 8 columns in my result. Not null columns will be first and null at the end.Here is a sample data :
I am using sql loader to load data to tables from tab delimeted files.
Here the problem is that the sql loader is not handling null values. If there is any null value in the flat file it is moving the field values to left and loading to the table. I am using NVL function to handle the null values, but it is not working.
My control file is: LOAD DATA INFILE 'C: da_poc_filesSQL_scriptsSourcefilesTRADEGLOBNODE1.TXT' BADFILE 'C: da_poc_filesSQL_scriptsBadfilesTRADEGLOBNODE1.bad' DISCARDFILE 'C:C: da_poc_filesSQL_scriptsDiscardfilesTRADEGLOBNODE1.dsc'
[Code]....
Source file is attached to this link For the attached file in the first record, tradedate value is coming in to tradeprice field
I have a table containing hundreds of columns and I would like to be able to qualify my select statements so that only those columns containing a value are returned. Something like:
Select (non null columns) from tablename where columnX = 'whatever'
We are working on a migration project and we need to move 75 million rows from source system to target system.
Total number of columns in source system - 90 cols.
Out of the 90 columns 10 cols are system fields and rest 80 are properties for each record.
We are required to migrate all system cols and some required properties. In total we will migrate around 25 columns[10+15] for each record.
Before actaul migration , we need to do a data cleansing activity and hence we move the data to a staging table.
To create the staging table, we considered the below appraoches.
1. Create the staging table with around 30 coloumns so as to fit the data from source system[map the columns based on datatype]
2.Create the staging table with actual columns[90 columns] and import only the required properties. The rest all columns will remain NULL.
Do the data cleansing and move to target system.
My question here is, if we go with approach 2, We will not mix the data, as there will be a one-to-one mapping. But many columns will not have data and remain NULL. Will it affect the performance since we deal with 75 million rows.
I have a repeating frame wich displays each item and its quantity and pricethe second frame is a fixed frame which will display the total and price after discount.both frames are enclosed in the same frame.I made space beteewn the two frames so the second frame always prints in the same placewhen I run the report the second frame appears in the second page and the first frame (items in invoice) appear in the first page.
I've tried to change the vertical and horizental properties of the frames but the report is still the same
I am experiencing somewhat same issue...but have been unable to resolve it(new to Oracle) I am getting the infile from flat file(data dump from SQL) using sqlldr to upload data to the Oracle table...since the data is already in the flat file...I cannot do anything in the SQL to pre-format the data...
Sample of ERROR I am getting - Column CREATE_DATE which has date and time - happens to other date time columns also if remove the CREATE_DATE from Control file(happens to every single line of record): ========================================== Record 2: Rejected - Error on table LGCY_CHS.METS_CHS_USER_PRIV, column CREATE_DATE. ORA-01841: (full) year must be between -4713 and +9999, and not be 0
Next week I will be getting an input file which will contain over 1000 data columns to be loaded into ORACLE. It's about 6,400 characters in length.
My question is...has ever created a huge ctl file like this to be used for SQLLoader, using so many columns? I will be sending certain columns(data) to certain tables, so it's not just going into 1. It will be about 6 tables.
From the previous forums i came to know about the "FILLER" option to skip any columns from loading into table.
In my case,i have columns say: col1,col2,col3,col4.....coln
Here i want to load only col3 and col4 and skip rest of the columns.I don't want to use FILLER option for each "n" number of columns to be skipped.Instead is there any options to load only the required 2 columns.
I am running a GROUP BY query on a few columns of enumerated data like:
select count(*), Condition, Size group by Condition, Size;
COUNT(*) CONDITION SIZE -------- ---------- -------- 3 MINT L 2 FAIR L 4 FAIR M 1 MINT S
Well, let's say I also have a timestamp field in the database. I cannot run a group by with that involved because the time is recorded to the milisec and is unique for every record. Instead, I want to include this in my group by function based on whether or not it is NULL.
For example:
COUNT(*) CONDITION SIZE SOLDDATE -------- ---------- -------- ---------- 3 MINT L ISNULL 2 FAIR L NOTNULL 2 FAIR M NOTNULL 2 FAIR M ISNULL 1 MINT S ISNULL
ID.................Date...........................number...period.....termination 434473......2012-04-18 00:00:00.000...............6.......month(s)...end of month 443080......2012-02-26 00:00:00.000...............36......month(s).....end of quarter
Now it's getting complicated. Date is the start date of a contract.
I need to calculate a date when the contract can be terminated and must compare it with the current date. Lets take the first row. In column "termination" is the termination date. A contract can be terminated at "end of month" (I need to check the last day of a month. In order to get the real termination date we need to subtract the column "number".
example 30.01.2012 - 7 month = 30.06. 2011
The thing I can do it, but my logic works only if the date is in the same year. I've been thinking a lot...sql is not important first....Have you got a theoretical solution?
I want to implement a business rule such as we have for each id at most 1 dat null. So, I've created this unique index on test.
create unique index x_only_one_dat_cess_null on test(id, case when dat_cess is null then 'NULL' else to_char(dat_cess, 'dd/mm/yyyy') end);
insert into test values (1, sysdate); insert into test values (1, sysdate - 1); insert into test values (1, null); insert into test values (1, null); -- ----- insert into test values (2, sysdate); insert into test values (2, sysdate - 1); insert into test values (2, null);
The 4th insert will cause an error and this is what I wanted to implement. OK. Now the problem is that for non-null values of dat, we can't have data like this
because of the unique index (the 2nd and the 3rd row are equal). So just for learning purposes, how could we allow at most one null value of dat and allow duplicates for non-null values of dat.
SQL> Describe Stu_Table Name Null? Type ----------------------------------------- -------- ---------------------------- STU_ID VARCHAR2(2) STU_NAME VARCHAR2(10) STU_CLASS VARCHAR2(10)
now when i try to modify this Stu_id column to not null its give me error.
SQL>ALTER TABLE Stu_Table MODIFY Stu_Id int(3)not null; ALTER TABLE Stu_Table MODIFY Stu_Id int(3)not null * ERROR at line 1: ORA-01735: invalid ALTER TABLE option
and when i try to add new column with not null its also gives me error
SQL> ALTER TABLE Stu_Table add C1_TEMP integer NOT NULL; ALTER TABLE Stu_Table add C1_TEMP integer NOT NULL * ERROR at line 1: ORA-01758: table must be empty to add mandatory (NOT NULL) column
I am trying user managed recovery using archived and redo log files. I had restored old cold backup, then copy latest control file, redo log files, and archived log files at actual locations. Now I tried to recover database using following steps.
Quote: connect sys as sysdba startup mount; recover database; auto
After applied some archived files, one of the archived file corrupted and recovery cancel due to error, then i shut down the database. Now i want to copy corrupted archived file from backup and run the recovery further. so my question that should i have to run recovery from the beginning or can i run the recover from the last status of the database.
ORA-06550: line 26, column 30: PL/SQL: ORA-06552: PL/SQL: Compilation unit analysis terminated ORA-06553: PLS-320: the declaration of the type of this expression is incomplete or malformed ORA-06550: line 7, column 6: PL/SQL: SQL Statement ignored
I am getting the error when i run the below anonymous block. Historic_report_data has got plenty of columns and using only the columns that are in tb_Statement in the query. It is working fine if i use a table with same number of columns as tb_statement instead of historic_report_data.
DECLARE v_cnt NUMBER:=0; v_data_load_error EXCEPTION; BEGIN dbms_output.put_line('Reconciliation scripts'); dbms_output.put_line('Reconcile tb_bank_Account_month and report_hist_data');
I was install oracle forms 11g r2 on windows 7 64 bit and it's work fine , but when i install oracle forms 11g r2 on windows 7 32 bit when i run my form this error appear
FRM-93652: The runtime process has terminated abnormally.
i trace my code , when i use global variable it's cause the problem , but when i don't use it the form run perfectly .
I've modified a report and I am able to run the SQL from toad and executed with no errors and output the results as expected. The SQL from the report is calling a function from DB. I granted execute permisiion to public. when I run the report, it has o errors, the result were fine. However, when another user run the report, it get the the following error.
Terminated with error: REP-300: "GET_NSCL_TOT": invalid identifier to_char ==> (get_NSCL_Tot('NFL',to_date(:period_end_date,'DD-MON-RRRR'),'1'),'999999.99')||','col2
Export: Release 10.2.0.3.0 - Production on Thu Dec 6 15:06:21 2012
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Connected to: Oracle Database 10g Release 10.2.0.3.0 - Production Export done in WE8MSWIN1252 character set and AL16UTF16 NCHAR character set server uses AL32UTF8 character set (possible charset conversion)
About to export specified users ...
EXP-00010: DISTESTA is not a valid username Export terminated successfully with warnings.
C:Documents and SettingsAdministrator>
When i connect from sqlplus, it doesn't give any erro. What could be the reason?
How do i map student_newId field to student_id field in STUDENT DB table so that new id should be inserted in student_id column. How do i specify the mapping in control file. I dont want to create a new column in student table. In control file i will specify the below, Is this a best approach?. Do we have any othe way?
