Skip to content

Oracle 11g – How to force a sql_id to use a plan_hash_value using SQL Baselines (saved from google cache )

Oracle 11g – How to force a sql_id to use a plan_hash_value using SQL Baselines (saved from google cache ) published on Комментариев к записи Oracle 11g – How to force a sql_id to use a plan_hash_value using SQL Baselines (saved from google cache ) нет




Oracle 11g – How to force a sql_id to use a plan_hash_value using SQL Baselines | rmoff















rmoff

June 28, 2011

Oracle 11g – How to force a sql_id to use a plan_hash_value using SQL Baselines

Filed under: etl, oracle, performance, plan management, sql plan baseline — rmoff @ 14:13

Here’s a scenario that’ll be depressingly familiar to most reading this: after ages of running fine, and no changes to the code, a query suddenly starts running for magnitudes longer than it used to.

In this instance it was an ETL step which used to take c.1 hour, and was now at 5 hours and counting. Since it still hadn’t finished, and the gods had conspired to bring down Grid too (unrelated), I generated a SQL Monitor report to see what was happening:

select DBMS_SQLTUNE.REPORT_SQL_MONITOR(
   type=>'HTML',
   report_level=>'ALL',sql_id=>'939abmqmvcc4d') as report
FROM dual;

(h/t to Martin Berger for this)

It showed a horrendous explain plan:

A very naughty plan

Using Kerry Osborne’s script to look at the plan_hash_value over time from AWR, it was clear that the CBO had picked a new, bad, explain plan.

So we knew the sql_id, and we knew the plan_hash_value of the plan which we wanted the CBO to use. But how to do this?

Back to Kerry Osborne again, and his article about SQL Plan Baselines. He (and others) write in detail about how and what SQL Plan Baselines are, but in essence it lets you tell Oracle which plan to use (or optionally, prefer) for a given sql_id.

Since the desired plan_hash_value was no longer in the cursor cache, we could get it back from AWR, loaded in via a SQL Tuning Set. Here’s the code with in-line comments explaining the function of each block:

/* 
Set up a SQL Baseline using known-good plan, sourced from AWR snapshots
https://rnm1978.wordpress.com/

In this example, sql_id is 939abmqmvcc4d and the plan_hash_value of the good plan that we want to force is 1239572551
*/

-- Drop SQL Tuning Set (STS)
BEGIN
  DBMS_SQLTUNE.DROP_SQLSET(
    sqlset_name => 'MySTS01');
END;

-- Create SQL Tuning Set (STS)
BEGIN
  DBMS_SQLTUNE.CREATE_SQLSET(
    sqlset_name => 'MySTS01',
    description => 'SQL Tuning Set for loading plan into SQL Plan Baseline');
END;

-- Populate STS from AWR, using a time duration when the desired plan was used
--  List out snapshot times using :   SELECT SNAP_ID, BEGIN_INTERVAL_TIME, END_INTERVAL_TIME FROM dba_hist_snapshot ORDER BY END_INTERVAL_TIME DESC;
--  Specify the sql_id in the basic_filter (other predicates are available, see documentation)
DECLARE
  cur sys_refcursor;
BEGIN
  OPEN cur FOR
    SELECT VALUE(P)
    FROM TABLE(
       dbms_sqltune.select_workload_repository(begin_snap=>22673, end_snap=>22710,basic_filter=>'sql_id = ''939abmqmvcc4d''',attribute_list=>'ALL')
              ) p;
     DBMS_SQLTUNE.LOAD_SQLSET( sqlset_name=> 'MySTS01', populate_cursor=>cur);
  CLOSE cur;
END;
/

-- List out SQL Tuning Set contents to check we got what we wanted
SELECT 
  first_load_time          ,
  executions as execs              ,
  parsing_schema_name      ,
  elapsed_time  / 1000000 as elapsed_time_secs  ,
  cpu_time / 1000000 as cpu_time_secs           ,
  buffer_gets              ,
  disk_reads               ,
  direct_writes            ,
  rows_processed           ,
  fetches                  ,
  optimizer_cost           ,
  sql_plan                ,
  plan_hash_value          ,
  sql_id                   ,
  sql_text
   FROM TABLE(DBMS_SQLTUNE.SELECT_SQLSET(sqlset_name => 'MySTS01')
             );

-- List out the Baselines to see what's there
SELECT * FROM dba_sql_plan_baselines ;

-- Load desired plan from STS as SQL Plan Baseline
-- Filter explicitly for the plan_hash_value here if you want
DECLARE
my_plans pls_integer;
BEGIN
  my_plans := DBMS_SPM.LOAD_PLANS_FROM_SQLSET(
    sqlset_name => 'MySTS01', 
    basic_filter=>'plan_hash_value = ''1239572551'''
    );
END;
/

-- List out the Baselines
SELECT * FROM dba_sql_plan_baselines ;

Now when the query’s run, it will use the desired plan.

Things to note:

  • In 10g and 11gR1 the default for SELECT_WORKLOAD_REPOSITORY is to return only BASIC information, which excludes the plan! So DBMS_SPM.LOAD_PLANS_FROM_SQLSET doesn’t load any plans.
    • It doesn’t throw a warning either, which it could sensibly, since the STS has no plan, and it can see that</grumble>
    • This changes to TYPICAL in 11gR2 (thanks Surachart!)
  • Parameter “optimizer_use_sql_plan_baselines” must be set to TRUE for a baseline to be used
  • Flush the cursor cache after loading the baseline to make sure it gets picked up on next execution of the sql_id

References:

Thanks to John Hallas for his help with this problem.

2 Comments

  1. very useful
    thx !

    Comment by djeday84 — July 21, 2011 @ 08:12

  2. […] не претендую на оригинальность, все взято отсюда и отсюда 55.755786 37.617633 Leave a Comment LikeBe the first to like this post. […]

    Pingback by perfomance « djeday84 — July 21, 2011 @ 10:37

RSS feed for comments on this post.

Sorry, the comment form is closed at this time.

The Silver is the New Black Theme. Blog at WordPress.com.




Follow

Get every new post delivered to your Inbox.

Join 33 other followers

















    how to vizualize table usage by queries with awr and exel

    how to vizualize table usage by queries with awr and exel published on Комментариев к записи how to vizualize table usage by queries with awr and exel нет

    generate pivot script:

    select listagg( ''''||sql_id||'''',',') within group (order by 1 )from (select distinct (sql_id) from  dba_hist_sql_plan where object_name='OPN_HIS' ) ;
    

    use info in awr to build a graph

    select  * from (
    select s.sql_id,
    sum( nvl(s.executions_delta,0)) execs,TO_CHAR (ss.begin_interval_time, 'DD.MM.YYYY HH24') date#
    -- sum((buffer_gets_delta/decode(nvl(buffer_gets_delta,0),0,1,executions_delta))) avg_lio
    from DBA_HIST_SQLSTAT S, DBA_HIST_SNAPSHOT SS, dba_hist_sqltext st
    where ss.snap_id = S.snap_id
    and ss.instance_number = S.instance_number
    and executions_delta > 0
    and elapsed_time_delta > 0
       and st.sql_id=s.sql_id
       and st.sql_text not like '/* SQL Analyze%'
       and s.sql_id in  ( select p.sql_id from dba_hist_sql_plan p where p.object_name='OPN_HIS') 
    and ss.begin_interval_time > sysdate-7
    group by TO_CHAR (ss.begin_interval_time, 'DD.MM.YYYY HH24'),s.sql_id )
    pivot ( sum(execs)  for sql_id in (
    '04chua8g507qc','0ppqkga09s858','13nb5fgqggy5y','2y61btbkgbt99','3akp3rn03dw0m','4hwvc5b6kg9s9','5btym214ffk95','5fjnq328z8cvm','5haf8w0xm1qrz','5yrbchx0n2ww9','70qkbhpxdyhmd',
    '7459pp1x54x2g','7hap4j3ar5fu8','9ym4u1trvzcq8','aabhxfrznx8ty','b4ynhjjwwdjf9','bp0sfgzv774u4','btm5kxhrc34qy','bxfhw0z4cnmxp','by4vbkw71nrax','dpnr7csjy3657','f3n5pm0ck9yuw','f452wsrxch6js' )
      );
    
    

    result :

    Screen Shot 2016-05-31 at 12.05.45

    script to find hidden param + description

    script to find hidden param + description published on Комментариев к записи script to find hidden param + description нет
    col "Parameter" for a50
    col "Session Value" for a10
    col "Instance Value" for a10
    col "Default value" for a5
    col SESSMOD for a7
    col SYSMOD for a9
    col "Description" for a95
    
    select a.ksppinm "Parameter", b.ksppstvl "Session Value", c.ksppstvl "Instance Value" ,b.ksppstdf "Default value", decode(bitand(a.ksppiflg/256,3),1, 'True', 'False') SESSMOD,decode(bitand(a.ksppiflg/65536,3),1,'IMMEDIATE',2,'DEFERRED',3,'IMMEDIATE','FALSE') SYSMOD,a.ksppdesc "Description" from sys.x$ksppi a, sys.x$ksppcv b , sys.x$ksppsv c where a.indx = b.indx and a.indx = c.indx and upper(ksppinm) like upper( '_%&1%') order by a.ksppinm;
    
    

    Script to capture and restore file permission in a directory

    Script to capture and restore file permission in a directory published on Комментариев к записи Script to capture and restore file permission in a directory нет
    Script to capture and restore file permission in a directory (for eg. ORACLE_HOME)

    [code]
    #!/usr/bin/perl -w
    #
    # Captures file permissions and the owner of the files
    # useage : perm1.pl <path to capture permission>
    #

    use strict;
    use warnings;
    use File::Find;
    use POSIX();

    my (@dir) = @ARGV;
    my $linecount=0 ;

    #print @ARGV, $#ARGV;

    if ($#ARGV < 0) {
    print "\n\nOpps….Invalid Syntax !!!!\n" ;
    print "Usage : ./perm1.pl <path to capture permission>\n\n" ;
    print "Example : ./perm1.pl /home/oralce\n\n" ;
    exit ;
    }
    my $logdir=$dir[0] ;
    #my ($sec, $min, $hr, $day, $mon, $year) = localtime;
    ##my ($dow,$mon,$date,$hr,$min,$sec,$year) = POSIX::strftime( ‘%a %b %d %H %M %S %Y’, localtime);
    my $date = POSIX::strftime( ‘%a-%b-%d-%H-%M-%S-%Y’, localtime);
    my $logfile="permission-".$date;
    my $cmdfile="restore-perm-".$date.".cmd" ;

    open LOGFILE, "> $logfile" or die $! ;
    open CMDFILE, "> $cmdfile" or die $! ;
    find(\&process_file,@dir);

    print "Following log files are generated\n" ;
    print "logfile : ".$logfile. "\n" ;
    print "Command file : ".$cmdfile. "\n" ;
    print "Linecount : ".$linecount."\n" ;
    close (LOGFILE) ;
    close (CMDFILE) ;

    sub process_file {
    my ($dev,$ino,$mode,$nlink,$uid,$gid,$rdev,$size, $atime,$mtime,$ctime,$blksize,$blocks,$username,$user,$pass,$comment,$home,$shell,$group);
    my %uiduname = () ;
    my %gidgname = () ;
    my $filename = $File::Find::name;

    #### Building uid, username hash

    open (PASSWDFILE, ‘/etc/passwd’) ;

    while ( <PASSWDFILE>) {
    ($user,$pass,$uid,$gid,$comment,$home,$shell)=split (/:/) ;
    $uiduname{$uid}=$user ;
    }
    close (PASSWDFILE) ;

    #### Building gid, groupname hash

    open (GRPFILE, ‘/etc/group’) ;

    while ( <GRPFILE>) {
    ($group,$pass,$gid)=split (/:/) ;
    $gidgname{$gid}=$group ;
    }
    close (GRPFILE) ;

    ($dev,$ino,$mode,$nlink,$uid,$gid,$rdev,$size, $atime,$mtime,$ctime,$blksize,$blocks) = stat("$filename");
    # printf "%o %s %s %s\n", $mode & 07777, $uiduname{$uid}, $gidgname{$gid}, $filename ;
    printf LOGFILE "%o %s %s %s\n", $mode & 07777, $uiduname{$uid}, $gidgname{$gid}, $filename ;
    printf CMDFILE "%s %s%s%s %s\n", "chown ",$uiduname{$uid}, ":", $gidgname{$gid}, $filename ;
    printf CMDFILE "%s %o %s\n", "chmod ",$mode & 07777, $filename ;
    # printf "%o %s %s %s\n", $mode & 07777, $uiduname{$uid}, $gidgname{$gid}, $filename ;
    $linecount++ ;
    }
    [/code]

    [свернуть]

    how to find what was in dba_2pc_pending transaction

    how to find what was in dba_2pc_pending transaction published on Комментариев к записи how to find what was in dba_2pc_pending transaction нет

    generate logminer list:

    select completion_time,'exec DBMS_LOGMNR.ADD_LOGFILE('''||name||''');' from v$archived_log where completion_time > trunc(sysdate) +9/24 and  completion_time < trunc(sysdate) +11/24 and dest_id=1;
    

    build catalog

    EXEC DBMS_LOGMNR.START_LOGMNR(   OPTIONS => DBMS_LOGMNR.DICT_FROM_ONLINE_CATALOG);
    

    find out transaction content:

    select global_tran_id,local_tran_id,l.* from (
     (
    select global_tran_id,local_tran_id,
    from DBA_2PC_PENDING) T
    JOIN  V$LOGMNR_CONTENTS L ON (l.xidusn||'.'||l.xidslot||'.'||l.xidsqn)=t.local_tran_id
    )
    

    commit\rollback:

    select ' commit force '''||local_tran_id||''';' from dba_2pc_pending ;
    --select ' rolback force '''||local_tran_id||''';' from dba_2pc_pending ;
    select 'exec DBMS_TRANSACTION.PURGE_LOST_DB_ENTRY('''||local_tran_id||'''); commit;' from dba_2pc_pending ;
    

    if archivelogs are on tape:

    select sum(blocks*block_size)/1024/1024/1024 gb_size, THREAD#, min(sequence#),max(sequence#) from gv$archived_log where dest_id=1 and first_time > trunc(sysdate) +9/24
    and first_time < trunc(sysdate) +11/24 group by THREAD# ;
    
    run {
    restore archivelog thread 1 from sequence 3925 until sequence 3938;
    }
    

    generate ash report from sqlplus

    generate ash report from sqlplus published on Комментариев к записи generate ash report from sqlplus нет

    plan to make a script with time range as input values

    19:22:41 (1)system@xxxxx> select * from table (dbms_workload_repository.ash_global_report_html( 4281370344,1,timestamp'2016-05-05 13:50:00',timestamp'2016-05-05 14:05:00') );
    

    open in browser:
    add for mac os x

    host open %SQLPATH%\tmp\output_&_connect_identifier..html
    

    add for linux:

    host firefox %SQLPATH%\tmp\output_&_connect_identifier..html --linux
    

    how to calculate avg row size including blobs

    how to calculate avg row size including blobs published on Комментариев к записи how to calculate avg row size including blobs нет

    NOTE: RUN ON STANDBY IF POSSIBLE =)

    alter session force parallel query parallel 4;
    set timing on
    set SERVEROUTPUT ON
    DECLARE
     l_vc2_table_owner VARCHAR2(30) := '&TABLE_OWNER';
     l_vc2_table_name VARCHAR2(30) := '&TABLE_NAME';
     /* sample number of rows */
     l_nu_sample_rows NUMBER := 1000000;
    
     /* loop through columns in the table */
     CURSOR l_cur_columns IS
     SELECT column_name, data_type FROM dba_tab_columns
     WHERE owner = l_vc2_table_owner
     AND table_name = l_vc2_table_name;
     l_rec_columns l_cur_columns%ROWTYPE;
     l_vc2_sql VARCHAR2(10000);
     l_avg_row_size NUMBER(10,2);
     BEGIN
     l_vc2_sql := '';
     OPEN l_cur_columns;
     FETCH l_cur_columns INTO l_rec_columns;
     /* loop through columns */
     WHILE l_cur_columns%FOUND
     LOOP
     /* if LOB datatype use dbms_log.get_length to find length */
     IF l_rec_columns.data_type = 'CLOB' OR l_rec_columns.data_type = 'BLOB' THEN
     l_vc2_sql := l_vc2_sql || 'NVL(dbms_lob.getlength(' || l_rec_columns.column_name || '), 0) + 1';
     ELSE
     l_vc2_sql := l_vc2_sql || 'NVL(VSIZE(' || l_rec_columns.column_name || '), 0) + 1';
     END IF;
     FETCH l_cur_columns INTO l_rec_columns;
     IF l_cur_columns%FOUND THEN
     l_vc2_sql := l_vc2_sql || ' + ';
     END IF;
     END LOOP;
     IF l_vc2_sql IS NOT NULL THEN
     l_vc2_sql := 'SELECT 3 + AVG(' || l_vc2_sql || ') FROM ' || l_vc2_table_owner || '.' || l_vc2_table_name
     || ' WHERE rownum < ' || l_nu_sample_rows;
     EXECUTE IMMEDIATE l_vc2_sql INTO l_avg_row_size;
     dbms_output.put_line(l_vc2_table_owner || '.' || l_vc2_table_name || ' average row length: ' || l_avg_row_size);
     ELSE
     dbms_output.put_line('Table ' || l_vc2_table_owner || '.' || l_vc2_table_name || ' not found');
     END IF;
     END;
     /
    
     

    how to add sql performance change metric to EM

    how to add sql performance change metric to EM published on Комментариев к записи how to add sql performance change metric to EM нет

    Hello, how to add custom metric to EM with “bad” sql and how-to diagnose results:
    usefull links first:
    Creating an Information Publisher Report based on a Custom Metric Extension

    whats_changed.sql

    based on this material add custom metric with query from
    query with performance degradation running more than 5 sec

    custom metrics 1
    Launch the ‘metric extension’ menu from: Enterprise/Monitoring/Metric Extensions
    custom metrics 2
    choose sql adapter
    custom metrics 3
    as results are multiple rows we need WA – add rownum
    custom metrics 4
    define columns from result set and alerts
    custom metrics 5
    credentials ( i use default )
    custom metrics 6
    run tests

    Screen Shot 2016-03-30 at 08.48.51

    custom metrics 7
    deploy metric on host

    after all
    Enjoy the message like this one:

    Host=xxxxxxxxx.local
    Target type=Database Instance
    Target name=SOME_DATABASE
    Message=The value of NORM_STDDEV is 16.668
    Severity=Critical
    Event reported time=Mar 29, 2016 3:00:55 AM MSK
    Operating System=Linux
    Platform=x86_64
    Associated Incident Id=19287
    Associated Incident Status=New
    Associated Incident Owner=
    Associated Incident Acknowledged By Owner=No
    Associated Incident Priority=None
    Associated Incident Escalation Level=0
    Event Type=Metric Alert
    Event name=ME$SQL_PERFOMANCE_CHANGE:NORM_STDDEV
    Metric Group=ME$SQL_PERFOMANCE_CHANGE
    Metric=NORM_STDDEV
    Metric value=16.66792070888098375763394835218401326307
    Key Value=
    Rule Name=all alarms by type:db and cluster,abushmelev
    Rule Owner=ABUSHMELEV
    Update Details:
    The value of NORM_STDDEV is 16.668
    Incident created by rule (Name = Incident management rule set for all targets, Create incident for critical metric alerts [System generated rule]).

    the metric is clickable and have historic information:
    Screen Shot 2016-03-30 at 08.54.30

    Screen Shot 2016-03-30 at 08.55.01

    based on this email with modification of Kerry Osborne’s script we can analyse what changed with sql_id=’0crfdgc01xcha’:

    09:21:24 (1)[p00esia]system@p00esia> @plan_hist2 0crfdgc01xcha
    old   8: where sql_id = '&1'
    new   8: where sql_id = '0crfdgc01xcha'
    
       SNAP_ID |   NODE | BEGIN_INTERVAL_TIME             | SQL_ID        | PLAN_HASH_VALUE |      EXECS |    AVG_ETIME |        AVG_LIO |    AVG_PIO
    ---------- | ------ | ------------------------------- | ------------- | --------------- | ---------- | ------------ | -------------- | ----------
         48097 |      1 | 16-MAR-16 11.40.46.963 PM       | 0crfdgc01xcha |      3594529397 |          1 |         .817 |          228.0 |          0
         48960 |      1 | 29-MAR-16 12.51.12.708 AM       | 0crfdgc01xcha |      3695145680 |          1 |        1.441 |          471.0 |         52
         48960 |      2 | 29-MAR-16 12.51.12.745 AM       | 0crfdgc01xcha                   |          1 |        1.299 |          233.0 |         27
         49010 |      1 | 29-MAR-16 05.40.02.717 PM       | 0crfdgc01xcha                   |          1 |        6.077 |      138,653.0 |       2073
         49053 |      1 | 30-MAR-16 08.00.11.253 AM       | 0crfdgc01xcha                   |          1 |        4.628 |       79,942.0 |       1199
         49054 |      2 | 30-MAR-16 08.20.13.680 AM       | 0crfdgc01xcha                   |          1 |        5.473 |      103,279.0 |       2643
    

    so, it’s clear that plan has changed and it run slower
    quick way to fix this – add baseline ( how-to )

    find query with performance degradation running more than 5 sec

    find query with performance degradation running more than 5 sec published on 2 комментария к записи find query with performance degradation running more than 5 sec

    find query with performance degradation running more than 5 sec :

    select * from (
    select sql_id, execs, before_avg_etime, after_avg_etime, norm_stddev,
           case when to_number(before_avg_etime) < to_number(after_avg_etime) then 'Slower' else 'Faster' end result
    from (
    select sql_id, sum(execs) execs, sum(before_execs) before_execs, sum(after_execs) after_execs,
           sum(before_avg_etime) before_avg_etime, sum(after_avg_etime) after_avg_etime,
           min(avg_etime) min_etime, max(avg_etime) max_etime, stddev_etime/min(avg_etime) norm_stddev,
           case when sum(before_avg_etime) > sum(after_avg_etime) then 'Slower' else 'Faster' end better_or_worse
    from (
    select sql_id,
           period_flag,
           execs,
           avg_etime,
           stddev_etime,
           case when period_flag = 'Before' then execs else 0 end before_execs,
           case when period_flag = 'Before' then avg_etime else 0 end before_avg_etime,
           case when period_flag = 'After' then execs else 0 end after_execs,
           case when period_flag = 'After' then avg_etime else 0 end after_avg_etime
    from (
    select sql_id, period_flag, execs, avg_etime,
    stddev(avg_etime) over (partition by sql_id) stddev_etime
    from (
    select sql_id, period_flag, sum(execs) execs, sum(etime)/sum(decode(execs,0,1,execs)) avg_etime from (
    select s.sql_id, 'Before' period_flag,
    nvl(s.executions_delta,0) execs,
    (s.elapsed_time_delta)/1000000 etime
    -- sum((buffer_gets_delta/decode(nvl(buffer_gets_delta,0),0,1,executions_delta))) avg_lio
    from DBA_HIST_SQLSTAT S, DBA_HIST_SNAPSHOT SS, dba_hist_sqltext st
    where ss.snap_id = S.snap_id
    and ss.instance_number = S.instance_number
    and executions_delta > 0
    and elapsed_time_delta > 0
       and st.sql_id=s.sql_id
       and st.sql_text not like '/* SQL Analyze%'
    and ss.begin_interval_time <= sysdate-1
    union
    select s.sql_id, 'After' period_flag,
    nvl(s.executions_delta,0) execs,
    (s.elapsed_time_delta)/1000000 etime
    from DBA_HIST_SQLSTAT S, DBA_HIST_SNAPSHOT SS ,dba_hist_sqltext st
    where ss.snap_id = S.snap_id
    and ss.instance_number = S.instance_number
    and s.executions_delta > 0
    and s.elapsed_time_delta > 0
       and st.sql_id=s.sql_id
       and st.sql_text not like '/* SQL Analyze%'
    and ss.begin_interval_time > sysdate-1
    )
    group by sql_id, period_flag
    )
    )
    )
    group by sql_id, stddev_etime
    )
    where norm_stddev > nvl(to_number('1'),2)
    and max_etime > nvl(to_number('5'),.1)
    )
    where result = 'Slower'
    order by norm_stddev;
    
    

    original by Kerry Osborne

    Primary Sidebar