Skip to content

PgSQL Indexes and “LIKE”

PgSQL Indexes and “LIKE” published on Комментариев к записи PgSQL Indexes and “LIKE” нет

Original from Paul Ramsey

Hi all, because English is not my native language, that’s why I will write as little as possible =)
here is the case, when we write a query with like and get Seq Scan (full table scan), instead index scan this material can be usefull for you

create table tt as   select s, md5(random()::text) from generate_Series(1,990000) s;
 create index on tt(md5);
 show LC_COLLATE;
 lc_collate
-------------
 en_US.UTF-8

 explain analyze select * from tt where md5 like 'a6b90b58a652b8e1bd01bbe2%';
                                               QUERY PLAN
--------------------------------------------------------------------------------------------------------
 Seq Scan on tt  (cost=0.00..20625.00 rows=4950 width=36) (actual time=132.559..132.559 rows=0 loops=1)
   Filter: (md5 ~~ 'a6b90b58a652b8e1bd01bbe2%'::text)
   Rows Removed by Filter: 990000
 Planning time: 0.203 ms
 Execution time: 132.583 ms

if we create same index but with text_pattern_ops we can get good improvement:

 create index on tt(md5 text_pattern_ops);

 explain analyze select * from tt where md5 like 'a6b90b58a652b8e1bd01bbe2%';
                                                    QUERY PLAN
------------------------------------------------------------------------------------------------------------------
 Index Scan using tt_md5_idx1 on tt  (cost=0.42..8.45 rows=99 width=37) (actual time=0.022..0.022 rows=0 loops=1)
   Index Cond: ((md5 >= 'a6b90b58a652b8e1bd01bbe2'::text) AND (md5 < 'a6b90b58a652b8e1bd01bbe3'::text))
   Filter: (md5 ~~ 'a6b90b58a652b8e1bd01bbe2%'::text)
 Planning time: 0.403 ms
 Execution time: 0.043 ms

ORA-42016: shape of interim table does not match specified column mapping

ORA-42016: shape of interim table does not match specified column mapping published on 1 комментарий к записи ORA-42016: shape of interim table does not match specified column mapping

try to make table partitioned:
create dummy table:

  CREATE TABLE USER."PERS_DATA_OPERATION_LOG_REDEF" 
   (	"ID" NUMBER NOT NULL ENABLE, 
	"OBJECT_OID" NUMBER NOT NULL ENABLE, 
	"OBJECT_SNILS" VARCHAR2(256), 
	"OBJECT_FULL_NAME" VARCHAR2(4000), 
	"SUBJECT_OID" NUMBER, 
	"SUBJECT_SNILS" VARCHAR2(256), 
	"SUBJECT_FULL_NAME" VARCHAR2(4000), 
	"CRT_ON" TIMESTAMP (6) , 
	"OPERATION_TYPE" VARCHAR2(256), 
	 CHECK (OPERATION_TYPE IN ('VIEW','PASS_RESET','REMOVE','BGIR_RESTART')) ENABLE, 
	 CONSTRAINT "PERS_DATA_OPERATION_LOG_PK1" PRIMARY KEY ("ID")
   ) 
     PARTITION BY RANGE ("CRT_ON")    INTERVAL(NUMTOYMINTERVAL(1, 'MONTH'))
     (partition p0 VALUES LESS THAN (TIMESTAMP' 2011-01-01 00:00:00') ) ;

start redef and get error:

exec dbms_redefinition.can_redef_table ('USER','PERS_DATA_OPERATION_LOG',dbms_redefinition.cons_use_pk);
   
exec  DBMS_REDEFINITION.start_redef_table ('USER','PERS_DATA_OPERATION_LOG','PERS_DATA_OPERATION_LOG_REDEF',NULL,DBMS_REDEFINITION.cons_use_pk);

ORA-42016: shape of interim table does not match specified column mapping

this is because original table have a timestamp with local timezone format
how to fix it:

select listagg(column_name,',') within  group  ( order by column_id) from dba_tab_columns where table_name='PERS_DATA_OPERATION_LOG';

begin 
DBMS_REDEFINITION.start_redef_table(
    uname        => 'USER',
    orig_table   => 'PERS_DATA_OPERATION_LOG',
    int_table    => 'PERS_DATA_OPERATION_LOG_REDEF',
    col_mapping =>'ID,OBJECT_OID,OBJECT_SNILS,OBJECT_FULL_NAME,SUBJECT_OID,SUBJECT_SNILS,SUBJECT_FULL_NAME,to_timestamp(crt_on) crt_on,OPERATION_TYPE',
    options_flag => DBMS_REDEFINITION.cons_use_pk);
end;



SET SERVEROUTPUT ON
DECLARE
 l_num_errors PLS_INTEGER;
BEGIN
 DBMS_REDEFINITION.copy_table_dependents(
   uname             => 'USER',
   orig_table        => 'PERS_DATA_OPERATION_LOG',
   int_table         => 'PERS_DATA_OPERATION_LOG_REDEF',
   copy_indexes      => DBMS_REDEFINITION.cons_orig_params, -- Non-Default
   copy_triggers     => TRUE,  -- Default
   copy_constraints  => TRUE,  -- Default
   copy_privileges   => TRUE,  -- Default
   ignore_errors     => true,
   num_errors        => l_num_errors);
 DBMS_OUTPUT.put_line('l_num_errors=' || l_num_errors);
END;
/
exec DBMS_REDEFINITION.SYNC_INTERIM_TABLE('USER','PERS_DATA_OPERATION_LOG','PERS_DATA_OPERATION_LOG_REDEF');

EXEC DBMS_REDEFINITION.finish_redef_table('USER', 'PERS_DATA_OPERATION_LOG', 'PERS_DATA_OPERATION_LOG_REDEF' );

Oracle 12c Adaptive Query Optimization overhead

Oracle 12c Adaptive Query Optimization overhead published on Комментариев к записи Oracle 12c Adaptive Query Optimization overhead нет

Trying to find out overhead of new Adaptive Query Optimization feature

use perf to find out process cpu usage ( eg:perf stat -e task-clock -p 9576 sleep 5 )
Screen Shot 2016-08-24 at 21.45.07

Screen Shot 2016-08-24 at 21.46.40
to disable adaptive features I used

alter system set optimizer_features_enable='11.2.0.1';

so the result is:

3000 hpares

3000 hpares

3000 hpares

3000 hpares

3000 hpares

avg

adaptive off

1724,46

1786,04

1696,22

1687,89

1690,89

1717,10

adaptive on

1803,79

1815,98

1966,07

1804,35

1819,35

1841,91

overhead

4,40

1,65

13,73

6,45

7,06

6,78

ps: lotshparses.sql link

How to resolve SQL object and column names all the way to base tables and columns in Oracle?

How to resolve SQL object and column names all the way to base tables and columns in Oracle? published on Комментариев к записи How to resolve SQL object and column names all the way to base tables and columns in Oracle? нет

original from Tanel Poder

was looking for source of v$restore point view, helped article from Tanel =)

 alter session set "_dump_qbc_tree"=1;

result is :

 SELECT  "ADDR","INDX","INST_ID","CON_ID","NRSNAME","NRSINCARN","NRSSCN","NRSTIME","NRSFLAGS","NRSRSPTIME","NRSRID","NRSSTM" FROM "SYS"."X$KCCNRS"   "X$KCCNRS" WHERE (con_id IN (0, 3) );

upd: also it can be done with:

 select view_definition from v$fixed_view_definition where view_name='GV$RESTORE_POINT';

script to find tables with nonindexed FK

script to find tables with nonindexed FK published on Комментариев к записи script to find tables with nonindexed FK нет

need to pass schema name as firs script argument

select table_name, column_name
              from ( select c.table_name, cc.column_name, cc.position column_position
                      from   all_constraints c, all_cons_columns cc
                      where  c.constraint_name = cc.constraint_name
                        and c.constraint_type = 'R' and c.owner=cc.owner and c.owner='&1'
                     minus
                     select i.table_name, ic.column_name, ic.column_position
                      from all_indexes i, all_ind_columns ic
                      where i.index_name = ic.index_name and i.owner=ic.table_owner and i.owner='&1');

Troubleshooting ‘enq: TX – index contention’ Waits (Doc ID 873243.1)

Troubleshooting ‘enq: TX – index contention’ Waits (Doc ID 873243.1) published on Комментариев к записи Troubleshooting ‘enq: TX – index contention’ Waits (Doc ID 873243.1) нет

today faced with ‘enq: TX – index contention’

Screen Shot 2016-08-05 at 21.08.15

awr shows

Screen Shot 2016-08-05 at 20.01.41

helped to make partition index on nonpartition table:

drop index ARADMIN.IT3060;
CREATE UNIQUE INDEX ARADMIN.IT3060_P ON "ARADMIN"."T3060" ("C1") global partition by hash (C1) partitions 16 parallel 16 ;
drop index ARADMIN.T3060_IDX_12;
CREATE index ARADMIN.T3060_IDX_12 ON "ARADMIN"."T3060"  ("C536870937", "C7", "C536870938", "C1") global partition by hash ("C536870937", "C7", "C536870938", "C1")  partitions 16 parallel 16 TABLESPACE "ARSYSTEM" ;
drop index ARADMIN.T3060_IDX_11;
CREATE index ARADMIN.T3060_IDX_11 ON "ARADMIN"."T3060"  ("C536870922", "C7", "C536870941") global partition by hash ("C536870922", "C7", "C536870941")  partitions 16 parallel 16 TABLESPACE "ARSYSTEM" ;

CRS-6706: Oracle Clusterware Release patch level (‘2222840392’) does not match Software patch level (‘0’). Oracle Clusterware cannot be started.

CRS-6706: Oracle Clusterware Release patch level (‘2222840392’) does not match Software patch level (‘0’). Oracle Clusterware cannot be started. published on Комментариев к записи CRS-6706: Oracle Clusterware Release patch level (‘2222840392’) does not match Software patch level (‘0’). Oracle Clusterware cannot be started. нет

faced with

COMMAND EXECUTION FAILURE :
Using configuration parameter file: /ora01/app/grid/12.1.0.2/OPatch/opatchautotemp_rac2/patchwork/crs/install/crsconfig_params
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac2'
CRS-2673: Attempting to stop 'ora.crf' on 'rac2'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac2'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac2'
CRS-2677: Stop of 'ora.mdnsd' on 'rac2' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'rac2' succeeded
CRS-2677: Stop of 'ora.crf' on 'rac2' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac2'
CRS-2677: Stop of 'ora.gipcd' on 'rac2' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-6706: Oracle Clusterware Release patch level ('2222840392') does not match Software patch level ('0'). Oracle Clusterware cannot be started.
CRS-4000: Command Start failed, or completed with errors.
2016/08/01 10:21:54 CLSRSC-117: Failed to start Oracle Clusterware stack

helped solution from:
CRS-1153: There was an error setting Oracle Clusterware to rolling patch mode. (Doc ID 1943498.1)
CRS-6706: Oracle Clusterware Release patch level (‘nnn’) does not match Software patch level (‘mmm’) (Doc ID 1639285.1)

first one:

[root@rac2 ~]# crsctl stop rollingpatch
CRS-1171: Rejecting rolling patch mode change because the patch level is not consistent across all nodes in the cluster. The patch level on nodes rac1 is not the same as the patch level [2369764000] found on nodes rac2.
CRS-4000: Command Stop failed, or completed with errors.

with no luck

second is successfull:

[root@rac1 ~]# kfod op=patchlvl
-------------------
Current Patch level
===================
2369764000
[root@rac1 ~]# kfod op=patches
---------------
List of Patches
===============
19396455
19769480
20299023
20831110
21359755
21436941
21948341
21948344
21948354


[root@rac2 ~]# kfod op=patches
---------------
List of Patches
===============
19396455
[root@rac2 ~]# kfod op=patchlvl
-------------------
Current Patch level
===================
2222840392

[root@rac1 ~]# sh /ora01/app/grid/12.1.0.2/crs/install/rootcrs.sh -patch
Using configuration parameter file: /ora01/app/grid/12.1.0.2/crs/install/crsconfig_params
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac1'
...
2016/08/01 10:57:54 CLSRSC-4015: Performing install or upgrade action for Oracle Trace File Analyzer (TFA) Collector.

2016/08/01 10:59:46 CLSRSC-4005: Failed to patch Oracle Trace File Analyzer (TFA) Collector. Grid Infrastructure operations will continue.
.....
Oracle Clusterware active version on the cluster is [12.1.0.2.0]. The cluster upgrade state is [NORMAL]. The cluster active patch level is [2369764000].

postgresql cat effective non default parameters from config file

postgresql cat effective non default parameters from config file published on Комментариев к записи postgresql cat effective non default parameters from config file нет
 cat postgresql.conf |egrep -v "^[[:blank:]]*#" | grep -v '^$'
data_directory='/u01/postgres/9.5/main/pgdata'
hba_file='/u01/postgres/9.5/main/conf/pg_hba.conf'
ident_file='/u01/postgres/9.5/main/conf/pg_ident.conf'
listen_addresses = '*'		# what IP address(es) to listen on;
max_connections = 300			# (change requires restart)
shared_buffers = 4096MB			# min 128kB
work_mem = 256MB				# min 64kB
maintenance_work_mem = 512MB		# min 1MB
dynamic_shared_memory_type = posix	# the default is the first option
checkpoint_timeout = 15min		# range 30s-1h
max_wal_size = 10GB
min_wal_size = 1GB
checkpoint_completion_target = 0.9	# checkpoint target duration, 0.0 - 1.0
archive_mode = on		# enables archiving; off, on, or always
archive_command = 'cp -i %p /u01/postgres/9.5/main/recovery/walarchive/%f </dev/null'		# command to use to archive a logfile segment
max_wal_senders = 3		# max number of walsender processes
max_replication_slots = 2	# max number of replication slots
hot_standby = on			# "on" allows queries during recovery
effective_cache_size = 8GB
log_destination = 'stderr'		# Valid values are combinations of
logging_collector = on			# Enable capturing of stderr and csvlog
log_directory = 'pg_log'		# directory where log files are written,
log_filename = 'postgresql-%Y-%m-%d.log'	# log file name pattern,
log_truncate_on_rotation = on		# If on, an existing log file with the
log_rotation_age = 1d			# Automatic rotation of logfiles will
log_rotation_size = 0			# Automatic rotation of logfiles will
log_min_duration_statement = 300	# -1 is disabled, 0 logs all statements
log_checkpoints = on
log_connections = on
log_disconnections = on
log_line_prefix = '%t [%p]: [%l-1] db=%d,appname=%a,user=%u,client=%h '			# special values:
log_lock_waits = on			# log lock waits >= deadlock_timeout
log_temp_files = 0			# log temporary files equal or larger
log_timezone = 'Europe/Moscow'
autovacuum_max_workers = 5		# max number of autovacuum subprocesses
autovacuum_vacuum_scale_factor = 0.01	# fraction of table size before vacuum
autovacuum_analyze_scale_factor = 0.05	# fraction of table size before analyze
datestyle = 'iso, mdy'
timezone = 'Europe/Moscow'
lc_messages = 'en_US.utf8'			# locale for system error message
lc_monetary = 'en_US.utf8'			# locale for monetary formatting
lc_numeric = 'en_US.utf8'			# locale for number formatting
lc_time = 'en_US.utf8'				# locale for time formatting
default_text_search_config = 'pg_catalog.english'

how to find which traces I can run

how to find which traces I can run published on Комментариев к записи how to find which traces I can run нет

from oracle-l@freelists.org

Here’s how you may find some of these yourself.

In this case you were looking for information on tracing locks, so this grep shows matching info from the messages file
(linux assumed here, with Gnu grep)

grep -E “^1[0-9]{4}.*\block” $ORACLE_HOME/rdbms/mesg/oraus.msg

10028, 00000, “Dump trace information during lock / resource latch cleanup”
10219, 00000, “monitor multi-pass row locking”
10621, 00000, “specify retry count for online index build cleanup DML lock get”
10623, 00000, “test synchronized flow of SORT rows into FOR UPDATE lock phase”
10629, 00000, “force online index build to backoff and retry DML lock upgrade”
10988, 00000, “event to get exclusive lock during materialized view refresh in IAS”
12766,00000, “ACMS failed to acquire share-mode mount lock”
12843, 00000, “pdml lock not held properly on the table”
13771, 00000, “cannot obtain exclusive lock %s on \”SQL Tuning Set\” \”%s\” owned by user \”%s\””
14403, 00000, “cursor invalidation detected after getting DML partition lock”
14700, 00000, “Object(s) owned by SYS cannot be locked by non-SYS user”
15003, 00000, “diskgroup \”%s\” already mounted in another lock name space”
15069, 00000, “ASM file ‘%s’ not accessible; timed out waiting for lock”
15150, 00000, “instance lock mode ‘%s’ conflicts with other ASM instance(s)”
16040, 00000, “Destination archive log file is locked.”
16415, 0000, “Event to enable lock dumping”
16641, 0000, “failure to acquire broker configuration metadata lock”

The output may be restricted to show only ‘lock’, and not include ‘locked’ or ‘locking’

grep -wE “^1[0-9]{4}.*\block” $ORACLE_HOME/rdbms/mesg/oraus.msg
10028, 00000, “Dump trace information during lock / resource latch cleanup”
10621, 00000, “specify retry count for online index build cleanup DML lock get”
10623, 00000, “test synchronized flow of SORT rows into FOR UPDATE lock phase”
10629, 00000, “force online index build to backoff and retry DML lock upgrade”
10988, 00000, “event to get exclusive lock during materialized view refresh in IAS”
12766,00000, “ACMS failed to acquire share-mode mount lock”
12843, 00000, “pdml lock not held properly on the table”
13771, 00000, “cannot obtain exclusive lock %s on \”SQL Tuning Set\” \”%s\” owned by user \”%s\””
14403, 00000, “cursor invalidation detected after getting DML partition lock”
15003, 00000, “diskgroup \”%s\” already mounted in another lock name space”
15069, 00000, “ASM file ‘%s’ not accessible; timed out waiting for lock”
15150, 00000, “instance lock mode ‘%s’ conflicts with other ASM instance(s)”
16415, 0000, “Event to enable lock dumping”
16641, 0000, “failure to acquire broker configuration metadata lock”

Keeping in mind that in Oracle lock has synonyms, such as enqueue

grep -wE “^1[0-9]{4}.*\benqueue” $ORACLE_HOME/rdbms/mesg/oraus.msg
10425, 00000, “enable global enqueue operations event trace”
10427, 00000, “enable global enqueue service traffic controller event trace”
10428, 00000, “enable tracing of global enqueue service cached resource”
10429, 00000, “enable tracing of global enqueue service IPC calls”
10433, 00000, “global enqueue service testing event”
10434, 00000, “enable tracing of global enqueue service multiple LMS”
10435, 00000, “enable tracing of global enqueue service deadlock detetction”
10437, 00000, “enable trace of global enqueue service S optimized resources”
10440, 00000, “enable global enqueue service inquire resource modes trace”
10666, 00000, “Do not get database enqueue name”
10706, 00000, “Print out information about global enqueue manipulation”
10862, 00000, “resolve default queue owner to current user in enqueue/dequeue”
10868, 00000, “event to enable interop patch for AQ enqueue options”
10999, 00000, “do not get database enqueue name”
12806, 00000, “could not get background process to hold enqueue”
16146, 00000, “%scontrol file enqueue unavailable”
19573, 00000, “cannot obtain %s enqueue for datafile %s”
19725, 00000, “can not acquire plug-in enqueue”

Still not finding anything useful, the -w parameter can be removed from grep to allow variations, such as ‘enqueued’ and ‘enqueues’

grep -E “^1[0-9]{4}.*\benqueue” $ORACLE_HOME/rdbms/mesg/oraus.msg
10425, 00000, “enable global enqueue operations event trace”
10427, 00000, “enable global enqueue service traffic controller event trace”
10428, 00000, “enable tracing of global enqueue service cached resource”
10429, 00000, “enable tracing of global enqueue service IPC calls”
10433, 00000, “global enqueue service testing event”
10434, 00000, “enable tracing of global enqueue service multiple LMS”
10435, 00000, “enable tracing of global enqueue service deadlock detetction”
10437, 00000, “enable trace of global enqueue service S optimized resources”
10440, 00000, “enable global enqueue service inquire resource modes trace”
10666, 00000, “Do not get database enqueue name”
10704, 00000, “Print out information about what enqueues are being obtained”
10706, 00000, “Print out information about global enqueue manipulation”
10862, 00000, “resolve default queue owner to current user in enqueue/dequeue”
10868, 00000, “event to enable interop patch for AQ enqueue options”
10999, 00000, “do not get database enqueue name”
12806, 00000, “could not get background process to hold enqueue”
16146, 00000, “%scontrol file enqueue unavailable”
19573, 00000, “cannot obtain %s enqueue for datafile %s”
19725, 00000, “can not acquire plug-in enqueue”

Since you are looking for trace (debug) specific information, a perusal of the oraus.msg file reveals:

/ Pseudo-error debugging events:
/ Error codes 10000 .. 10999 are reserved for debug event codes that are
/ not really errors.

So searches can be further restricted by changing the range of numbers to search for, showing only debug events:

grep -E “^10[0-9]{3}.*\benqueue” $ORACLE_HOME/rdbms/mesg/oraus.msg
10425, 00000, “enable global enqueue operations event trace”
10427, 00000, “enable global enqueue service traffic controller event trace”
10428, 00000, “enable tracing of global enqueue service cached resource”
10429, 00000, “enable tracing of global enqueue service IPC calls”
10433, 00000, “global enqueue service testing event”
10434, 00000, “enable tracing of global enqueue service multiple LMS”
10435, 00000, “enable tracing of global enqueue service deadlock detetction”
10437, 00000, “enable trace of global enqueue service S optimized resources”
10440, 00000, “enable global enqueue service inquire resource modes trace”
10666, 00000, “Do not get database enqueue name”
10704, 00000, “Print out information about what enqueues are being obtained”
10706, 00000, “Print out information about global enqueue manipulation”
10862, 00000, “resolve default queue owner to current user in enqueue/dequeue”
10868, 00000, “event to enable interop patch for AQ enqueue options”
10999, 00000, “do not get database enqueue name”

Notice the use of -i now to ignore case

> grep -iE “^10[0-9]{3}.*sql\*net” $ORACLE_HOME/rdbms/mesg/oraus.msg
10079, 00000, “trace data sent/received via SQL*Net”

That particular event was one I found in the messages file a couple years ago, and it was instrumental in discovering and issue with SqlNet, thin clients and lobs.
The issue was far too much network traffic, proved out by this trace event.
The solution was to use a 12.1.0.2 thick client.

Jared Still
Certifiable Oracle DBA and Part Time Perl Evangelist
Principal Consultant at Pythian
Pythian Blog http://www.pythian.com/blog/author/still/
Oracle Blog: http://jkstill.blogspot.com
Home Page: http://jaredstill.com

Primary Sidebar