Change Data Capture For Java Processes And Not Databases?
Mar 11, 2013
oracle 11.2.0.3 I have insert only tables that receive srecords from multiple processes at a rate of about 200/second. Each transaction can have up to 100 records. I have another set of processes that queries this table for the latest data. These processes run anywhere from once a minute to once an hour. Processes do not get all of the data. They get data based on a type field.
Both of these are from java middle tiers. The process that queries data (The subscriber) does so at the request of many remote servers (there will be vast numbers). I am not allowed to expose these downstream databases to the internet (they are not oracle DBs anyway) so I cannot use streams or golden gate
So basicallyInsert Process: multiple sessions that combined insert records up to 200/second. There will be between 1-100 records per commit.
Query Process: Downtream process makes a request to my middle tier. This middle tier runs a query to get the latest data and passes it back. This design is set and I cannot change it.
1. right now we capture the insert time of the record. However, at this rate of inserts some processes will commit faster than others. So I cant use a 'greater than my insert time' query.
2. streams/golden gate won't work. can't register these DBs.
3. don't want to serialize my inserts because since I am not sure I can keep up with the insert rate. I don't even know what the specs will be for the production hardware. I have to actually deliver this before its decided. So I am being conservative.
4. I really want to avoid updates on this table if possible. In part due to my limited ability to test.
5. due to the number of downstream processes it is possible that it will request data and for some reason fail to insert the data locally. So the downstream application will keep track of the latest data it received. This means that a subscriber may need to request the same data again.
Is there a way to set up change data capture with multiple subscribers to handle this? if my subscribers are just queries? All the queries come from the same servers(there will be several, but all the same thing). If so, when I performance test this are there any wait issues I should keep an eye on?
View 1 Replies
ADVERTISEMENT
Jan 31, 2013
I have big source tables to load in a data warehouse. We are in a full Oracle environment. So I need to extract only delta since the last extract.
I need to capture even deleted rows from the source table.
I have tested the following solution:
- declare a materialized view log on the source table
- load the content on this view log in my ODS
- empty this view log
- load my DWH with the captured delta
It is very simple and seems to work perfectly. I am just confused by the fact that nobody seems to have implemented such a solution.
View 11 Replies
View Related
Jan 6, 2011
I wonder is there any way to capture all the invalid nodes in the xml when the xml is validated with an xsd (schema).
I am using java's xml validation package to perform validation (javax.xml.validation packages in java 1.5)
View 1 Replies
View Related
Nov 15, 2012
Is there any way I can capture on change event for the Select List (already created on the page) using java script and then refresh chart region?
View 2 Replies
View Related
Aug 15, 2013
How to change Character set for oracle 8i database. Is there anyway to change the Character set without affecting the current database.
View 11 Replies
View Related
Mar 27, 2012
I have the task to migrate the total databases(Exact copy to be moved to another server).The current server is going for format.After I did the following steps I am getting the tablespaces(databases)-4 sizes same ,but I am facing issue like some default tablespaces i.e temp,system are not matching.
temp tablespace
***************
current server - 4.0(approximately)
Migrating server - 160 MB
System tablespace
*****************
current server - 580 MB
Migrating server - 220 MB
Also I checked the tables are also matching for the 4 databases.Also Provide the solution or method which is correct.
steps done for migrating(By me)
********************************
EXPORTING DATA USING DATAPUMP
*********************************
1 From command prompt MKDIR 'c:oraclexeapp mp';
2 From SQL prompt conn system/kotak;
3 create or replace directory dmpdir as 'c:oraclexeapp mp';
4 grant read,write on directory dmpdir to kotak;
5 From command prompt
expdp system/kotak@xe full=Y directory=dmpdir dumpfile=xe.dmp logfile=expdpxe.log;
IMPORTING DATA USING DATAPUMP
*****************************
in another server machine
1 From SQL prompt conn system/kotak;
2 create or replace directory dmpdir as 'c:oraclexeapp mp';
3 grant read,write on directory dmpdir to kotak;
4 From command prompt set ORACLE_SID=xe;
5 impdp system/kotak@xe full=Y directory=dmpdir dumpfile=xe.dmp logfile=impdpxe.log;
IN OUR PROCESS we created the below tablespaces and user before IMPORTING created 4 tablespaces
1. kotak
2. kotakdb
3. wired_data
4. ferrari
Created Users which are there in 219 server
1. KOTAK
2. KOTAKDB
3. FC_80
4. DEMOINTERNETBANK
5. DEMOINTERNETBANKDB
6. CREDITCARD
View 1 Replies
View Related
Oct 12, 2012
I want copy some tables from one database to another. Which of following is the fastest way:
- INSERT /*+APPEND*/ via database link
- Transportable Tablespaces
- datapump IMP/EXP (I supposed it's the slowest way)
View 2 Replies
View Related
Oct 4, 2011
I want to compare the table contents across the two different databases , one is on netezza and other is on Exadata.
View 2 Replies
View Related
Nov 22, 2012
We want to replicate the data between the databases.We have 4 databases in a network.If there will be any change in database 1,e.g. updation in any table,it should automatically replicate on other 3 databases.or user will change something in database 2 ,it should replicate on other 3 databases and vice versa. All 4 databases have same schema and same configuration.
View 1 Replies
View Related
Mar 28, 2011
we want to use database link to connect a Database for operating the select,update or ... commands,our destination database is WE8ISO8859P1 and current database is AR8MSWIN1256 cahrset, but when we operate a command to view data,all NonEnglish characters appear odd wich we can not recognize the appeared text, also if we use convert function no change would make, view right charachters with our database link.
it doesnt work with convert founction
select convert(menu_name,'US7ASCII','WE8ISO8859P1'),
convert(menu_name,'ar8mswin1256','WE8ISO8859P1'),
convert((convert(menu_name,'US7ASCII','WE8ISO8859P 1')),'ar8mswin1256','WE8ISO8859P1'),
menu_name
from T$R_MENU@"TO201.US.ORACLE.COM"
WHERE MENU_ID=601011;
result is
EU?iY ?C?ICa? OCOaI?
E???? ?C?IC?? ?C??I?
EU?iY ?C?ICa? OCOaI?
E???? ?C?IC?? ?C??I?
View 1 Replies
View Related
Apr 18, 2009
I am current using Pro*C to upload data from a flat file to database.
Now I have to transfer information(data) between databases. As a rookie, I do not know an efficient of way doing it.
View 4 Replies
View Related
Jun 1, 2010
1) how to to compare sybase table data with oracle database table data?
2) how to connect sybase database from oracle database?
View 5 Replies
View Related
Mar 30, 2011
I have one primary database server and one physical standby database serve. but i am unable to fix "ORA-16607" while enabling configuration. during log switch redo data is being applied to the phy standby side but some thing wrong happens in BROKER
here are details........about my configuration
Primary database name PRIM
Physical Database Name STAN
net service PRIM
net service STAN
DGMGRL> show configuration
Configuration
Name: test
Enabled: YES
Protection Mode: MaxPerformance
Fast-Start Failover: DISABLED
Databases:
prim - Primary database
stan - Physical standby database
Current status for "test": Warning: ORA-16607: one or more databases have failed
======================================================
Alert log file First para show no problem for for redo to be transmitted to stndby
======================================================
LNS1 started with pid=48, OS id=5408
Thu Sep 21 21:32:03 2006
Thread 1 advanced to log sequence 59
Current log# 1 seq# 59 mem# 0: /u01/app/oracle/oradata/PRIM/redo01.log
[code]......
View 2 Replies
View Related
Sep 29, 2010
I have one primary database and one physical standby database on my data guard environment.
Now i want one more physical standby database in my data guard environment, meaning i want 2 physical standby. since this is a production environment, i need to take great care. However, downtime will be mandatory.
[u]init.ora for production:[/u]
pup2.__db_cache_size=6241124352
pup2.__java_pool_size=33554432
pup2.__large_pool_size=134217728
pup2.__oracle_base='/oracle/app/oracle'#ORACLE_BASE set from environment
pup2.__pga_aggregate_target=5771362304
[code]......
init.ora for my first primary standby
pup2.__db_cache_size=6509559808
pup2.__java_pool_size=33554432
pup2.__large_pool_size=67108864
pup2.__oracle_base='/oracle/app/oracle'#ORACLE_BASE set from environment
pup2.__pga_aggregate_target=5502926848
[code].......
View 5 Replies
View Related
Apr 9, 2013
I have recently configure Data guard with Database 10g (10.2.0.4-64 bits) on Windows 2007 server.My Data Gurad Configuration show Success status with 2 databases on same (or local) location.My questions are
1-When I query SHOW PARAMETER LOG ARCH
DG_CONFIG(PRMDB)
ONLY 1 (PRIMARY DATABASE IS DISPLAYED ONLY NOT 2 DTATABASES
e.g. DG_CONFIG(PRMDB, STLDB)
2- How to check the log applied interval or time (either transaction by transaction, timing etc)
View 11 Replies
View Related
Nov 14, 2013
How can find the number of standby databases configured for primary database from os level
View 9 Replies
View Related
Oct 21, 2012
Our organization has recently decided to go for storage metro cluster solution for disaster recovery. In a Data guard environment, we normally calculate how much archive log is generating and based on that value we calculate the required bandwidth.
For storage metro cluster, we need to find how much block is changing in our primary database, and the same rate of change would apply on DR cluster. Now, i need to give the assumption how much changing is happening in my system. How to calculate the change.
View 4 Replies
View Related
May 6, 2013
how to migration from ODI to java.. How to work on this type of migration.
View 1 Replies
View Related
Dec 20, 2010
I want to retrieve values from a XML data stored on CLOB data type and store it in different table.
Retrieving data from Soap Envelope and Soap Body?
<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope>
<soap:Header>
<company>US</company>
</soap:Header>
<soap:Body>
Data
</soap:Body>
<Account>ABCD1234</Account>
</soap:Envelope>
View 8 Replies
View Related
Mar 18, 2013
Below is the XML i have :
<?xml version="1.0" encoding="UTF-8" ?>
- <Transaction xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.test.com/Support/Services/test1/2012" xsi:schemaLocation="http://schemas.test.com/Support/Services/test1/2012 Support.test1.v1.xsd">
- <Request>
<Memid>10</Memid>
<Actid>32</Actid>
[code].......
I want to extract the element values :
The below code will be useful when i have no namespace , what must be done in order to work with namespace and extract element value
v_string_xml :=
' //Request[' || TO_CHAR (counter_xml) || ']/Memid/text()';
v_ssp_table (v_ssp_table.COUNT).memid :=
p_xml_in.EXTRACT (v_string_xml).getnumberval ();
v_string_xml :=
[code].......
View 2 Replies
View Related
Feb 9, 2012
How to extract the data from XML using the xsd file. attached files.
Explanation: first check the EmailMessage tage from order_conf.xml compared with Email.xml(<xsd:element name="EmailMessage">) if exists then go to next node.
EmailMessage(exists tag in order xml file)
->next <ns1:emailNotificationype> this tag should be follow under the EmailMessage tag(<xsd:element ref="emailNotificationype">) in Email.xml
->next <ns1:orderNotification> -> check this tag in <xsd:element name="orderNotification"> in Email.xml.
-> next <ns1:templateFormatInfo> -> it should follow under <xsd:element name="orderNotification"> in Email.xml.
-> next <ns1:templateFormatInfo> -> it should follow these tages <xsd:element name="templateFormatInfo"> <xsd:element ref="templatecode"/>
<xsd:element ref="templateversion"/>
i have to extract the value in between tags.
templatecode -> ORDCONF
templateversion ->1.0
Finally i have to load the data into table.
View 17 Replies
View Related
Jul 10, 2012
I've generated XML from oracle:
select dbms_xmlquery.getxml('select * from tbl_nm') from dual
<?xml version="1.0" encoding="ISO-8859-1"?>
<ROWSET>
<ROW num="1">
<NAME>admin</NAME>
<PASSWORD>a</PASSWORD>
[code]....
i tried to get this xml data in html table but the table was not displayed. When I changed the <ROW> tag into any other tag say <CD> the table was displayed.
View 10 Replies
View Related
Jan 12, 2011
I have to parse an XML and store the data in a table.
My XML :-
WITH data AS (
SELECT XMLTYPE(
'<?xml version=''1.0'' encoding=''UTF-8''?>
[Code].....
The output is :-
CONVERSATIONID COMMAND VALUE
a949-9614-4e9c-9f49-831ab6cc6a41 OPEN_BROWSER V1
There will be cases when I will get multiple values for the columns conversationid, command, value.
<?xml version=''1.0'' encoding=''UTF-8''?>
<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/">
<S:Body>
[Code].....
In that case the output should be as :-
CONVERSATIONID COMMAND VALUE
a949-9614-4e9c-9f49-831ab6cc6a41 OPEN_BROWSER V1
iiii-9614-4e9c-9f49-831ab6cc6a41 OPEN_BROWSER V2
Not sure how to go about XML parsing when I have multiple values.
View 4 Replies
View Related
Mar 25, 2013
I have two databases (say DATABASE 1 with user USER1 with table A and DATABASE2 with user USER1 with table A). I want to select the table A in USER1 DATABASE2 from DATABASE1 USER1.
View 5 Replies
View Related
Jan 6, 2011
I want to send data from my window based standalone application to my web based application.
View 1 Replies
View Related
Jun 26, 2011
how to use dbms_xmlgen to take the results of a query and convert it into a XML formatted data. But if a column (clob) contains data in an XML format, is there a way to query that column and easily parse the data? I know that I could write loops that do string searches for each of the XML tags and nested tags.
View 9 Replies
View Related
Oct 14, 2013
Our company database is Oracle based and we use SQL Developer to pull out needed data.Using a snippet borrowed from a co-worker, I have put together a query that, among other things, pulls a list value out of an xml clob field and displays it in the query results. My query as it stands right now is below, followed by an example snippet of the xml clob that I am pulling from. The reason for the "query within a query" is because the base query could return multiple entries and I only want the one with the most recent date.
select * from
(Select Wtr_Service_Tag, Wtr_Tran_Origin, Wtr_Send_Date, Wtr_Receive_Date,
to_char(substr(wtr_req_xml,instr(substr(wtr_req_xml,1,8000),'SID')+8,12)) Asset_Tag
from ws_transactions
Where Wtr_Service_Tag In ('20458749610')
And Wtr_Req_Xml Like ('%CSM%')
Order By Wtr_Receive_Date Desc)
where rownum = 1;
[code]....
This query is only able to pull the first value in the list.How can I edit this query to pull all of the list items when there are more than 1 (preferably concatenated into one column in the query results)? I have another field, in a separate table, that I can pull from to get the number of list items.
This one may be more complex. As currently written, the query pulls a fixed number of characters from the xml clob and either returns not enough data, or too much because the values I need to pull could be of varying lengths. I have no way to query what those lengths might be.
View 28 Replies
View Related
Jan 24, 2013
I have a table Table_xml_tmp which has a column MESSAGECONTENTS as XMLType data which contains data as follows
"<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<xclimSchemaMain xmlns="http://clones.telcordia.com">
<response>
<clHeader>
<msgName>queryPlace</msgName>
[code]......
I need to get all rows where node <operationStatus> value as SUCCESS. I am running following query but it is returning null value insted of data of columns
SELECT a.MESSAGECONTENTS.extract('/clHeader/msgName/object/msgType/sender/senderInstance/operationStatus/text()').getStringVal()
FROM Table_xml_tmp a
WHERE a.MESSAGECONTENTS.extract('/clHeader/msgName/object/msgType/sender/senderInstance/operationStatus/text()').getStringVal()= 'SUCCESS'
View 2 Replies
View Related
Jul 9, 2012
CREATE TABLE EMP(NAME VARCHAR2(10 BYTE))
INSERT INTO EMP VALUES ('C');
INSERT INTO EMP VALUES ('A');
INSERT INTO EMP VALUES ('T');
SELECT xmlelement("NAME",NAME) FROM EMP;
I am trying to convert column data to xml format, but I get this error message:
Quote:The query fails because all columns types are currently not supported. I am using:
Quote:Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bi
PL/SQL Release 10.2.0.4.0 - Production
SQLTools 1.5.0 Beta build 9 as EDITOR
View 7 Replies
View Related
Feb 15, 2011
I have two separate databases, I want to send XML file contains a query from one to the other , that read query and return results to a first via XML file.
View 10 Replies
View Related