April 2008


 

There are various ways to Find the Duplicates in a table,but the tough thing is to identify the correct one. so here we will discuss about the differents methods to find the duplicates in a table.

Enabling Unique constraint

The fastest way of detecting duplicate records in a table is to attempt to switch on a unique constraint.
so lets create a Test Table and do with an example by adding some duplicate data.

SQL> create table TestDup
(
empno number,
ename varchar2(20),
sal number,
deptno number
);Table created.SQL> Insert into TestDup values(1000,’A',5000,10);1 row created.

SQL> Insert into TestDup values(1001,’B',8000,20);

1 row created.

SQL> Insert into TestDup values(1000,’A',9000,20);

1 row created.

SQL> commit;

Commit complete.

Now we will try to add a Unique constraint to the Table TestDup and see
what happens. So i am creating a Unique Constraint on combination of
empno and ename column. Note: Unique Constraint will be not created as
long as duplicate rows (based on empno and ename) are there in the table.


SQL> Alter table TestDup add Constraint U_Const_01 Unique(empno,ename);
ERROR at line 1:
ORA-02299: cannot validate (SCOTT.U_CONST_01) – duplicate keys found
So it is throwing an Error saying that the table contains duplicate rows and
so unique constraint cannot be applied. so now comes the part to capture the
duplicate rows. First of all we need a table to capture the duplicate rows,
the table creation script is available in the ORACLE_HOME installation path,
as follows

$ORACLE_HOME\rdbms\admin\utlexcpt.sql

I executed the following create table command from the sql file.


SQL> Create Table Exception_table(row_id rowid,
owner varchar2(30),
table_name varchar2(30),
constraint varchar2(30));
Table created.
now we are ready to catch the duplicate keys. so now i will again add the
Constraints but with a little in the syntax;

 

 

SQL> Alter table TestDup add Constraint U_Const_01 Unique(empno,ename) exceptions into Exception_table;
ERROR at line 1:
ORA-02299: cannot validate (SCOTT.U_CONST_01) – duplicate keys found
so if you see the syntax, i was using a keyword called EXCEPTIONS INTO , to redirect the duplicate keys
into the table Exception_Table. so now to see the duplicate rows, we use the following query


SQL> select * from TestDup where rowid in (select row_id from Exception_table);
EMPNO    ENAME  SAL        DEPTNO
 

 

———- ———- ———- ———- 
1000            A      5000      10
1000            A      9000      20

so it gives the two rows which are duplicate. so now we can take corrective action to remove
the duplicates and then add the constraint to the table to avoid further duplication.

 

 

NULLs affect IN and EXISTS

An IN is functionally equivalent to the = ANY clause , which means an OR Condition.

SQL> Select 1 From Dual where 1 in (1, NULL);1
———-
1


Because True or NULL will return True.
Same Exists is also same as IN..It returns True even if NULL is returned by the Subquery.Exists checks whether a row exists..it doesn’t check whether it is NULL or Not.

 

SQL> select ‘true’ val from dual where exists (select null from dual);VAL
—-
true


The difficult part comes when you use NOT IN.
The NOT IN clause is virtually equivalent to comparing each value with = and failing if any test is FALSE or NULL. It is equivalent to AND of all conditions, even if one of the condition is false or NULL, it wont return.

 


SQL> select ‘true’ from dual where 1 not in (null,2);no rows selected



if you understand the functionalities of IN , EXISTS and NOT IN then you can avoid common errors.

 

Undocumented Functions

sys_op_map_nonnull() is a undocumented Function to make NULL values equal.
It does not affect the Normal Comparisons.

SQL> Select 1 From Dual Where sys_op_map_nonnull(‘A’) = sys_op_map_nonnull(‘A’);1
———-
1

SQL> Select 1 From Dual Where sys_op_map_nonnull(NULL) = sys_op_map_nonnull(NULL);

1
———-
1


(Cont..)

Aggregate Functions

Aggregate Functions like SUM , COUNT will ignore NULL vaules and processes the NON NULL values alone.

SQL> Create table TestSumNull(a Number);Table created.

SQL> Insert into TestSumNull values(16);

1 row created.

SQL> Insert into TestSumNull values(5);

1 row created.

SQL> Insert into TestSumNull values(NULL);

1 row created.

SQL> Commit;

Commit complete.

SQL> Select * from TestSumNull;

A
———-
16
5

 

SQL> Select NVL(A,0) From TestSumNull;NVL(A,0)
———-
16
5
0

SQL> Select Sum(A) , Count(A) from TestSumNull;

SUM(A)   COUNT(A)
———- ———-
21             2

you can see from above result that the COUNT function has ignored the NULL value.

SQL*PLUS Commands

There is a SQL*PLUS command SET NULL .
so the String what is given will be displayed in case a NULL is returned (Alternative for NVL in a SQL*PLUS environment alone).


SQL> Set NULL EMPTY

SQL> Column Value Format a10

SQL> Select Null “Value” From Dual;

Value
———-
EMPTY

as you can see “Empty” is displayed instead of a NULL (without using NVL)…

 

(Cont…..)

Logical Operation

When you use NULL in Logical Operations, it has some intersting result but logical.
Note : Boolean is not a Valid Data Type in SQL…it is valid only in PL/SQL

The Truth table is as follows

OP1 Operator OP2 Result
True AND NULL NULL
False AND NULL False
True OR NULL True
False OR NULL NULL

So Logically Thinking we can come to the conclusion why two of the above four is returning Boolean.

For example, false and null is false.
This makes sense because null, being an unknown value, could in this this context either be true or false.
False And => putting Unknown as True or False…the Result is always False
hence false and null is certainly false as well.

On the other hand, false or null is null because the result is true for false or true and false for false or false, hence the expression’s value is unknown, or null.

note: In the following Example, Since Boolean is not supported directly in SQL, i have manipulated it using Relational Operators.


In the above query, ‘1 = 1′ will return True and (1 > NULL) returns NULL ….since True or NULL is True,
whatever is there in the Select is returned.

SQL> Select ‘TRUE’ Res From Dual Where (1 = 1) OR (1 > NULL);

RES
—-
TRUE

 


SQL> Select ‘False’ Res From Dual Where Not ((1 > 1) and (1 > NULL));

The above query is for False and Null Returns False..so applying a NOT Operator will make it True and ouput is returned

RES
—–
False

NULL, the magic word in the database world. NULL means nothing or unknown.

It is not equal to space in case of varchar or zero in case of number.
Oracle treats the empty string (”) as null.

This is not ANSI SQL Standard.The length of an empty string(”) is null, not 0.

SQL> select length('') from dual;

LENGTH('')
----------

SQL> select NVL(to_char(length('')),'NULL') Len From Dual;

LEN
----
NULL

Arthimetic Operation

Any Arthimetic Operation(+,-,*,/) will return NULL if any one of the operand is NULL.

SQL> Select NVL(To_Char(1+2+3+4+5+NULL),'NULL') Sum from Dual;

SUM
----
NULL

Note :

NVL(NULL Value Handling function) is a Oracle Built in function to Convert NULL into some Meaning full value.
Example: NVL(expr1,expr2) => if expr1 is NULL then it Returns Expr2 Else it return expr1.

NVL2 is another function to Handle NULL values.
Example : NVL2(expr1,expr2,expr3) => if expr1 is NOT NULL then it returns expr2 else if expr1 is NULL then Returns expr3.

Relational Operator

Relational Operators (> , < , >= , <=, <>) will Return NULL if any one of the operand is NULL.so if you want to find the number of rows which has a particular column value as NULL. you cannot use ‘=’ operator to check. You need to use a Special operator ‘IS‘ to check for NULL values.

SQL> Select 1 From Dual Where NULL = NULL;

no rows selected

SQL> Select 1 From Dual Where NULL is NULL;

1
----------
1

SQL> Select count(*) from Emp where Comm is NULL;

Count
-----
8

You can generate a matrix table which shows all operations by the following code snippet,

SQL> Create Table Operand
2 (
3 num number
4 );

Table created.

Insert Non NULL and NULL Values into the table.

SQL> insert into operand values ( 11);

1 row created.

SQL> insert into operand values ( 42);

1 row created.

SQL> insert into operand values (null);

1 row created.

SQL> Commit;

Commit complete.

The Query to Generate the matrix is as follows

ANSI SQL Syntax :

Select op_left.num "op left",
case when op_left.num = op_right.num then ' = '
when op_left.num > op_right.num then ' > '
when op_left.num < op_right.num then ' < '
else ' ? '
end "rel",
op_right.num "op right"
From
operand op_left CROSS JOIN operand op_right;

The Output is as follows
op left rel op right
------- --- --------
11       =    11
11       <    42
11       ?   EMPTY
42       >    11
42       =    42
42       ?   EMPTY
EMPTY    ?    11
EMPTY    ?    42
EMPTY    ?   EMPTY

9 rows selected.

Deleting Files with a Pattern or Extension within a folderIf you wish to delete all files or a certain file within a folder and all subfolders,

 

1. simply go to Start -> Run -> “cmd.”
2. When the command prompt appears, go to the directory that you wish to delete from.
using cd command. For example

c:\>cd d:\kamal


3. Type in

D:\Kamal> DEL /F /S [*.type]

For example, if I wanted to delete all MP3 files from my PC, i would type

D:\Kamal> DEL /F /S *.mp3

 All of the MP3 files on my D:\Kamal folder will be deleted.

Note that if you do not want to delete files within subfolders, don’t type the “/S.”

               Everyone is interested in the result of an SQL statement, here we are going to see what is happening in the background when a SQL is executed.

                      During the parsing phase, Oracle will break down the submitted SQL statement into its component parts, determine what type of statement it is (Query, DML, or DDL), and perform a series of checks on it. The first two steps in parsing are

1. Syntax Check

             Oracle checks that the SQL statement is valid. it checks for the rightness of the keywords and position. For example, whether SELECT statement has FROM in right position?

2. Semantic Check

           Semantic Check checks if the statement is valid in the context of the objects in the database like Do the tables and columns referenced in the SQL statement actually exist in the database? Does the user executing the statement have access to the objects and are the proper privileges in place? Are there ambiguities in the statement? For example, consider a SQL Select statement “Select * from EMP123″, syntactically the statement is right, but if there is no table called EMP123, then it is semantically wrong. 

For example, the following SQL statement fails with a syntax error:

  SQL> select from where 4;
select from where 4
*
ERROR at line 1:   ORA-00936: missing expression

Here is an example of a SQL statement that fails with a semantic error:

  SQL> select * from table_doesnt_exist;
select * from table_doesnt_exist
*
ERROR at line 1:   ORA-00942: table or view does not exist