Performance Tuning :: Best Practices To Optimize A Fast Refresh Materialized View
Jan 16, 2012
I have a base table with ~20 mio. records with two FAST REFRESH Materialized Views based on that table using various aggregate functions in their view definition.
The problem is, when e.g. one record changes in the base table, I see two records in the MV log table MLOG$, but invoking the fast refresh mechanism by using using:
1) TXN_HD_FCT (This table has 6 billion records. Has fields TXN_DT, CUST_ID, TXN_AMT, TXN_ID and the table is partitioned on TXN_DT. We load data approximately 600K+ records to this table every day 2) RELINK_CUST (This table has 2 fieds CUST_ID and NEW_CUST_ID). We load data approximatley 100-200K+ records every day into this table. This table has total 10MM+ records now.
Now i need to create a table (TXN_UPD_HDR_FCT) in Schema B by joining above 2 tables using below query every day
CREATE TABLE TXN_UPD_HDR_FCT select a.TXN_DT, nvl(b.new_cust_id, a.cust_id) as cust_id, a.txn_amt, a.txn_id, from schema_A.TXN_HD_FCT a, schema_A.RELINK_CUST b where a.cust_id = b.cust_id(+);
We update CUST_ID to NEW_CUST_ID based on some rules (and keep it in the table RELINK_CUST) and need to take this new_cust_id while building the table TXN_UPD_HDR_FCT.
Building this table is taking approximately 4+ hours every day. just refresh only records from TXN_HDR_FCT whose CUST_ID is changed or any records inserted into TXN_HDR_FCT? I read about Materialized views but not sure how can i create MV logs on 2 tables? If i use materialized views do i need to create indexes every day?
I have created a materialized view and also a normal View, which has 3 tables used in both the views, when inserted new records it reflects in a normal view but when i select the materialized view i cant see the updated data.
here is the materialized view i created;
CREATE MATERIALIZED VIEW pct_sales_materialized BUILD IMMEDIATE REFRESH ON DEMAND ENABLE QUERY REWRITE AS SELECT A.DEP_NAME,B.EMP_ID,C.EMP_NAME FROM department_head A,department_child B,emp_detail C WHERE A.DEP_ID = B.DEP_ID AND B.EMP_ID = C.EMP_ID
i have created a fast refresh materialized view but it is not performing fast refresh or even complete refresh.Then i have checked my materialized view capabilities then in mv_capabilities_table i got a message that materialized view log is newer than old .
I'm trying to build up a materialized view with aggregate and FAST REFRESH for INSERT and UPDATE, DELETE with no success. But the web doesn't deny it ?
We are planning to create a materialized view on 5 tables and 2 views. After first complete refresh we want to do Fast refresh going forward and planning to create view log for base tables. Since the SQL uses two views, we can create view-log for its underlying tables. But will that work ? In case of fast refresh ? or it will still go for complete refresh ?
The customer moved their database from 10g to a 11gR2 database.They created the MV in the new system and now it takes 26 hours to refresh as opposed to 15 min in the old 10g database.Just looking for a game plan for troubleshooting.
I have a view that includes sub queries in its select.
I am creating a MV over that view. and i want to fast refresh it when ever a commit is placed on the table (that is used in the from clause for the simple view)
create materialized view ino_crm.MV_IDENTIFICATION_DETAILS REFRESH FAST as SELECT billing_account_id, i.s1 KEYS_ , COALESCE(i.s2, i.b, i.d) VALUES_ FROM ba_personal_details pd , xmltable ( '//entry[string = "identification Details"]//entry' passing xmltype(pd.details) columns s1 VARCHAR2 (255) path 'string[1]' ,s2 VARCHAR2(255) path 'string[2]' ,b VARCHAR2(255) path 'boolean' ,d VARCHAR2(255) path 'date' ) i WHERE pd.mandator_key = 'VSO01' AND pd.valid_to = to_date('01.01.4000', 'DD.MM.YYYY'); and
I get an exception like: SQL Error: ORA-12015: cannot create a fast refresh materialized view from a complex query 12015. 00000 - "cannot create a fast refresh materialized view from a complex query"
*Cause: Neither ROWIDs and nor primary key constraints are supported for complex queries. *Action: Reissue the command with the REFRESH FORCE or REFRESH COMPLETE option or create a simple materialized view.
We are facing serious problem while refreshing materialized views using fast refresh option in ORACLE..For the very first time we are performing Complete refresh of data from DB1 to DB2 for few tables.Ongoing we are performing Fast Refresh.Sometimes the fast refresh works fine without any error and sometimes it fails with the below error.
ERROR at line 1: ORA-32320: REFRESH FAST of "CIR"."C_BO_COMM" unsupported after cointainer table PMOPs ORA-06512: at "SYS.DBMS_SNAPSHOT", line 803 ORA-06512: at "SYS.DBMS_SNAPSHOT", line 860 ORA-06512: at "SYS.DBMS_SNAPSHOT", line 841 ORA-06512: at line 1
let me know when will this error occur during Fast Refresh.
I'm trying to create materialized view, successfully create logs for all the tables involved.
CREATE MATERIALIZED VIEW LOG ON tbJournal WITH ROWID, SEQUENCE (tabid, companyid, storelocid, tabstatus, linetype) INCLUDING NEW VALUES; CREATE MATERIALIZED VIEW LOG ON TBTABS WITH ROWID, SEQUENCE (TSTAMP, COMPANYID, NUMGUESTS, POSITIONID, STORELOCID, TABSSTATUS) INCLUDING NEW VALUES;
[code]....
The result is ora-12015: cannot create a fast refresh materialized view from a complex query
It does work fine if I remove
AND EXISTS (SELECT 1 FROM tbJournal J WHERE j.tabstatus NOT BETWEEN -6 AND -4 AND j.linetype = 1 AND T.tabid = J.tabid AND T.companyid = J.companyid AND T.storelocid = J.storelocid) from the where clause
We have a MV which fetches data from around 27 tables containing 26 joins out of which 25 are outer joins. Some tables in the query are being referred multiple times through different alias names and hence the actual no of physical tables used is 18. This MV takes about 50 mins to refresh through complete refresh mechanism. We decided to make it fast refresh and thus made these configurations:
- Created MV logs based on rowid for each of the base tables. - Recreated MV using FAST refresh,with primary key option enabled - Pulled rowid for all these tables in the select column statement.
Even after making all the recommendations suggested by Oracle for fast refresh MV's we are still getting refresh time of around 65 mins(refresh time increased!!!).We already have indexes built on all the join columns of the base tables. What else do we need to do to make this a "fast" refresh MV ?
So I was reading about indexes here:[URL]...1Is there any reason to NOT use an index? If there aren't, then should you use an index on every column on every table?
What is the general best practice with indexes? After reading the section, it seems that there are only positive impacts of using an index, so why are they not automatically created?
CREATE Materialized View log on t1 with rowid; CREATE Materialized View log on t2 with rowid; CREATE Materialized View log on t3 with rowid; CREATE Materialized View MV_test BUILD IMMEDIATE FAST REFRESH ON COMMIT [code].......
I have created a Materialized View by joining the above 3 tables. The Materialized View logs with ROWID has been created on each of the 3 base tables. One of the base tables has around 160 million rows and i expect the Materialized View to have the same number of rows.
The Materialized View has been built for FAST REFRESH on COMMIT. The SELECT list of the CREATE Materialized View statement includes all the ROWIDS from each base tables for doing the fast refresh.
The DBMS_MVIEW explain view shows that the view is capable of doing fast refresh on any DML. The Materialized View took 3 hrs to get built. I have also created indexes on all the three rowid columns (t1rowid,t2rowid,t3rowid) on the Materialized View.
Now, When i do any update on one of the base table and do a commit,
UPDATE t1 set value=500 where tx_id='1234';
The commit is taking around around 15 minutes. I have generated the trace and had a look at it, and it shows that, to refresh the Materialized View, ORACLE is first deleting that record from the Materialized View and then inserting the new record from the base table to the Materialized View. This delete done by oracle is being done on the basis of the rowid of the updated record stored on the Materialized View log,
However its not using the index created on the t1rowid of the Materialized View and doing a full scan of the MV and hence the delete is taking a lot of time. The delete statement is using a hint NO_MERGE and HASH_SJ which is causing it to do the full table scan of the MV. why is ORACLE not using the index on the t1rowid column of the MV to do this delete?
Also to do the insert in the MV, its selecting the new changed value from the base tables t1,t2 and t2. Here again its doing a full table scan of the base table t1, rather then using the rowid directly or the index coulmn.which is taking a lot of time.
I have the source tables having 10 columns. I want to track the delta changes of only 6 columns. In case of insertion/updation, that new record should be inserted into another DELTA table of different schema.While in case of deletion, A new row will be inserted into that DELTA table , with the value of the primary key and other columns populated with NULL values.
I have applied the concept of 'after trigger' specifying the columns names but it logic is not working properly. Its tracking the changes for other 4 columns also.
I have to convert some existing materialized views (fast refresh) to partition materialized views.
Database version is oracle 10.1.0.4. I have decided to use on prebuilt table option to do the partitioning as it minimizes the time to transfer from the master site.
1) stop replication 1) create interim tables with similar structure as the materialized views 2) transfer all data from the materialized views to the interim tables 4) script out the materialized views structure and add in on prebuilt table option in the scripts 5) drop the materialized views 6) rename the interim tables with the same name as the materialized views 7) run the scripts to create the materialized views with on prebuilt table option 8) refresh the newly created materialized views -> it should take a short time since I am using on prebuilt table option
But I am facing one major issue. That is if I drop the materialized views, the materialized view logs of the master tables are purged. When the materialized views are refreshed fast, there are some data missing. the data that are purged out when the materialized view are dropped.
Do you happen to know other ways that existing materialized views can be converted to partitioned materialized views? Do you have any workaround to prevent the materialized view logs from being purged?
I have an issue in materialized view which has got one of the null able column and query on this column taking approximately 2 mins where as other indexed columns takes less than 10 sec.
Here is the summary
SQL> Select Count (1), Count (VAT_NO) From Mv_customer;
If an index is created on VAT_NO will that improve the performance. What kind of index can be created considering very less number of records has got VAT_NO
CREATE MATERIALIZED VIEW LOG ON ABC; CREATE MATERIALIZED VIEW MV_ABC REFRESH FAST START WITH SYSDATE NEXT SYSDATE+20/(24*60) FOR UPDATE AS SELECT * FROM ABC WHERE TMSTP> SYSDATE-1;
It is working fine. Appended the Query with WHERE clause in MV_ABC MVIEWS.
DROP MATERIALIZED VIEW LOG ON ABC; DROP MATERIALIZED VIEW MV_ABC; CREATE MATERIALIZED VIEW LOG ON ABC; CREATE MATERIALIZED VIEW MV_ABC REFRESH FAST START WITH SYSDATE NEXT SYSDATE+20/(24*60) FOR UPDATE AS SELECT * FROM ABC WHERE TMSTP> SYSDATE-1;
ORA-12013: updatable materialized VIEWS must be simple enough TO DO fast refresh
In search queries generally we select 10-25 columns (more can't be displayed on the screen) from 5-10 tables
Say in case of insurance related application, the search might be on policy number, policy holder's first name, policy holder's last name, region, policy type etc.
And not to many columns we are displaying on the screen, say, 4 tables have collectively 4 * 20 = 80 columns, then we are displaying say 12-15 columns with 2-3 columns have aggregates on it.
since the search criteria (e.g. first name, last name, policy number etc.) is not known till last moment it will be a generic dynamic query
Is it possible that instead we create a Materialized view with query with only joining conditions but no filter conditions and selecting only columns to be displayed on the screen and then we will refresh the materialized view (to take care of recent business transactions) and fire refined query with filter criteria on this materialized view
Select col1,col2,col3,col4,col5 From tab1,tab2,tab3,tab4 Where tab1.col1=tab2.col1 And tab2.col2=tab3.col2 And tab2.col2=tab4.col2;
Will it improve performance of the search functionality
I am not able to create materialized view with 'Refresh Fast on Commit' when the table is residing on the another user environment, I am not able to find the reason why?
You can re-generate the scenario at your end with the following code-
--Login to System- create user manu identified by manu; grant connect, resource to manu; create user arpit identified by arpit; grant connect, resource to arpit; grant create materialized view to manu;
[code]....
but when I am doing it with on demand options, materialized view is being created-
--Login to Manu- create materialized view arpit_view build immediate refresh on demand as select * from arpit.arpit /
I created a Materialized view using joining of two tables with FAST REFRESH ON COMMIT option. These tables are the main tables of the DB and contains alot of data(about 15 millions of records). two master tables have index but mview doesn't have any indexes. I've thought the entire concept of the fast refresh is that it should be a relatively quick operation but the result showed that DB performance reduced, for example I could insert 900 records per minute thru application into master tables before but now I can just insert about 400 records per hour!.Also, It slows down the select operation, went from 0.11 seconds(with joining two tables) to 33 seconds (in mview)!.
First of all, I considered following possibilities and test them:
1) The most likely solution was that a complete refresh was happening. However, the materialized view refresh was confirmed to be a fast refresh by querying USER_MVIEWS.
2) I checked the query of the materialized view and confirmed that it was a simple select from the master tables without any aggregation or sorting.
3) DBMS_MVIEW.explain_mview was run on the materialized view. MV_CAPABILITIES_TABLE did not reveal any problems.
4) A query of dba_registered_snapshots showed that no other snapshots appeared to be using these logs. So, After each refresh, the records in mview logs were deleting.
As you see, it seems everything is ok, but what was causing this fast refresh to go so slow?
I use oracle 10.2 on Suse Linux enterprise server.
I am having two server A and B.B is the replica of A.I have to alter the materialized view of one of the table in server B,of which Master table is in both the servers .I do not have any Materialized view log on both the servers and want to convert the Refresh method from Complete to Fast refresh.One way of this altration is to create materialized view log on master table.
A scheduler job is there to refresh some materialized views. All the views are run under single job name. Some mviews are not refreshing and those are not refreshing in the next run too.if we run those mviews manually then they are working fine from then . after some days those views which i told before are not refreshing and this happens frequently and i check the job_run_details views also and i am not seeing any errors in that.
I created a materialized view which takes around 6 hrs to refresh. way to reduce the refresh time.The query is:
create materialized view ERC_DAILY_DISBURSMENT_MIS_VIEW refresh complete on demand as with disb_qur as (select e.account_no, e.journal_dt, e.instr_amt disbursed_amt,
I have attempted to set up the materialized view below to refresh complete at the time specified below. The problem is that it does not refresh and pick up the changes to one of the base tables.
how to Refresh materialized view by other user than owner.option is there to give drop any view/table grant to other user. But can't do this as it's a system privileges.
Is there any option to refresh a view without dropping the structure?
Have some issue with Materialized view refresh. Have presented scenario which we have .
Schema1 : 1. Have base tables 2. Created MV logs on base tables
Schema2 : 1. Created sysnonyms for Schema1 base tables 2. created Materialized views 3. Created MV group , added views to the group
Executing the below command in schema2 : execute dbms_refresh.refresh('MV_grp');
Error : ORA-12048: error encountered while refreshing materialized view "Schema2"."Table1_MV" ORA-12018: following error encountered during code generation for "Schema2"."Table1_MV "ORA-00942: table or view does not exist ORA-06512: at "SYS.DBMS_SNAPSHOT", line 2255 ORA-06512: at "SYS.DBMS_SNAPSHOT", line 2461 ORA-06512: at "SYS.DBMS_IREFRESH", line 683 ORA-06512: at "SYS.DBMS_REFRESH", line 195 ORA-06512: at line 1
We have 2 production (OLTP) database( DB1 (18GB) and DB2 (200GB) ) on a server out of which DB2 database has almost 20 Materalized Views all (REFRESH FORCE ..ON DEMAND).
I have already informed the client that this is not the OLAP database where you are going to make use of the M Views for reporting purpose. However currently the client does not have any alternative.
Breif details :- Out of 20 M Views 3 refresh once a week and where as the rest refresh every night daily.
We are currently testing a new website for which we had to re-schedule all M Views from their Old timings to New Timings i.e. Daily Basis.
But post re-scheduling the database performance was badly degarded as the database was too slow also the number of archives increasted to 300GB which was earlier 100 GB.To resolve the issue , we had to go back to the old timings and the database is currently working fine.
Requirement :-Client is looking out for an alternative solution as he needs to once again change the M View refresh time i.e. Daily Refresh.We already have 2 Oracle Instance on the server
Questions :-
1. Can i create another database ( sayTESTDB2 ) in no archivelog mode and create a db link to reduce the load on the server ? 2. If Yes .. Then is there any relavent link /example or Metalink Note ID for the same?? 3. What all parameters do i need to consider while creating a DB Link ? 3. Out of 32 GB RAM , I have given 4GB and 8GB sga_max to DB1 and DB2 Respectively . Also the cpu utilization is normally 50-%60%.
Since currently we do not have any Licence for other server .. Can i make use of the same server ( i.e. creating 3 rd Oracle Instance )?
Since the main intention of this activity is reduce the load from the DB2 onto TESTDB when the daily refresh has set once again
