One of the most commonly asked question I get when explaining how the ZDLRA works has to do with Block Change tracking.
The question is usually something like...
"If the the Block Change Tracking file only keeps 7 days of history by default, how does it work with the ZDLRA where you only perform a full backup once"?
Well let's walk through what the block change does,from a high level.
If you want the detail of the internals, Alex Gorbachev wrote a great MOS note explaining it all.
ORACLE 10G BLOCK CHANGE TRACKING INSIDE OUT (Doc ID 1528510.1)
I'm going to start with a simple example.
First I'm going to turn on block change tracking, and then check the File# for my database files.
I am going to use the internal structure X$KRCFBH to view what's in the block change tracking.
Let's see what's in it when we start after running some updates
I can see that there are entries for all my tablespaces, with a "version count" of 1 and low/high time of 0.
Now I am going to execute a full backup and see what happens.
You can see that the high SCN set to 8362001.
Now I am going to look at the RMAN backup (USERS tablespace) to see what the Checkpoint SCN number was for the backup.
Ahhh.. Now I can see how it fits together. The high SCN for the BCT file is SCN number right before the checkpoint taken with the backup.
Now let's execute an incremental backups and see what happens.
By looking at the SYSTEM tablespace (file 1) I can see exactly what is happening with the BCT file.
The first version marks the starting SCN prior to the first backup after creating the file.
The second version marks the checkpoint SCN of the first backup (low), and the SCN prior to the second backup(high).
These marks, LOW/HIGH SCN, can be used to identify the blocks that changed between the backups.
Now I am going to perform a few more incremental backups with a few changes to the USERS tablespace and see what happens to the SYSTEM and USERS tablespaces versions.
First here is my query.
col tablespace_name format a10
select (select tablespace_name from dba_data_files where file_id=fno) tablespace_name,vercnt,to_char(vertime,'mm/dd/yy hh24:mi:ss') vertime,low, high from x$krcfbh
where fno in (select file_id from dba_data_files)
order by 1,2;
Let's see what my backups look like for File 7 (USERS tablespace)
My backups for both of these match.
Notice that I performed an Incremental level 0 backup, and then 8 Incremental level 1 backups.
I actually performed both differential backups and cumulative backups but it didn't matter.
Now let's look at the block change tracking file for these 2 tablespaces (system and users)
Very interesting.. Since I only made few changes to the users tablespace it has 3 versions, the oldest of which is the full backup.
The system tablespace has gone over 7 versions it no longer has the original version from the level 0 backup.
Now let's see if it used the BCT files for the backups.
WOW... Notice that the system tablespace (FILE #1) could not use the BCT file for the last backup, but the backup of the user tablespace (FILE #7) could because there were no changes between a few of the backups.
I can also see that the high SCN for users, 8364636,is older than the high SCN for system.
I am going to change users, and perform an incremental to see what happens to the LOW/HIGH scn in the next version.
After the backup, there is no gap in the SCN numbers. It created a new version that contained the changes between the previous version HIGH and the SCN of the incremental backup.
I also performed a Level backup of the database, and since USERS didn't change, a new version wasn't created in the BCT for the USERS tablespace.
1) The BCT file keeps a bit map of changes between 2 SCN numbers
2) The BCT file keeps changes on a file level.
3) If a file didn't change between backups, it doesn't create a new record in the BCT file. It doesn't matter if the backup is an incremental or full backup.
4) By default 7 versions are kept for EACH DATAFILE. After 7 days of backups, some datafiles may still use the BCT if they haven't changed.
The question is usually something like...
"If the the Block Change Tracking file only keeps 7 days of history by default, how does it work with the ZDLRA where you only perform a full backup once"?
How does the BCT work ?
Well let's walk through what the block change does,from a high level.
If you want the detail of the internals, Alex Gorbachev wrote a great MOS note explaining it all.
ORACLE 10G BLOCK CHANGE TRACKING INSIDE OUT (Doc ID 1528510.1)
I'm going to start with a simple example.
First I'm going to turn on block change tracking, and then check the File# for my database files.
SQL> alter database enable block change tracking using file '/home/oracle/app/oracle/oradata/BSG18/bct.dbf';
SQL>
TABLESPACE_NAME FILE_ID
------------------------------ ----------
USERS 7
UNDOTBS1 4
SYSTEM 1
SYSAUX 3
I am going to use the internal structure X$KRCFBH to view what's in the block change tracking.
Let's see what's in it when we start after running some updates
select (select tablespace_name from dba_data_files where file_id=fno) tablespace_name,vercnt,to_char(vertime,'mm/dd/yy hh24:mi:ss') vertime,low, high from x$krcfbh
where fno in (select file_id from dba_data_files);
SQL> SQL> 2
TABLESPACE VERCNT VERTIME LOW HIGH
---------- ---------- ----------------- ---------- ----------
SYSTEM 1 07/29/19 09:21:17 0 0
SYSAUX 1 07/29/19 09:21:18 0 0
UNDOTBS1 1 07/29/19 09:21:18 0 0
USERS 1 07/29/19 09:21:18 0 0
I can see that there are entries for all my tablespaces, with a "version count" of 1 and low/high time of 0.
Now I am going to execute a full backup and see what happens.
TABLESPACE VERCNT VERTIME LOW HIGH
---------- ---------- ----------------- ---------- ----------
SYSTEM 1 07/29/19 09:21:17 0 8362001
SYSAUX 1 07/29/19 09:21:18 0 8362001
UNDOTBS1 1 07/29/19 09:21:18 0 8362001
USERS 1 07/29/19 09:21:18 0 8362001
You can see that the high SCN set to 8362001.
Now I am going to look at the RMAN backup (USERS tablespace) to see what the Checkpoint SCN number was for the backup.
File LV Type Ckp SCN Ckp Time Abs Fuz SCN Sparse Name
---- -- ---- ---------- ----------------- ----------- ------ ----
7 0 Incr 8362002 07/29/19 09:46:59 NO /home/oracle/app/oracle/oradata/BSG/datafile/o1_mf_users_fz01zl49_.dbf
Ahhh.. Now I can see how it fits together. The high SCN for the BCT file is SCN number right before the checkpoint taken with the backup.
Now let's execute an incremental backups and see what happens.
TABLESPACE VERCNT VERTIME LOW HIGH
---------- ---------- ----------------- ---------- ----------
SYSAUX 1 07/29/19 09:21:18 0 8362001
SYSTEM 1 07/29/19 09:21:17 0 8362001
UNDOTBS1 1 07/29/19 09:21:18 0 8362001
USERS 1 07/29/19 09:21:18 0 8362001
SYSAUX 2 07/29/19 09:46:59 8362001 8363961
SYSTEM 2 07/29/19 09:46:59 8362001 8363961
UNDOTBS1 2 07/29/19 09:46:59 8362001 8363961
1 0 Incr 8362002 07/29/19 09:46:59 NO /home/oracle/app/oracle/oradata/BSG/datafile/o1_mf_system_fz01xnqo_.dbf
1 1 Incr 8363962 07/29/19 10:04:00 NO /home/oracle/app/oracle/oradata/BSG/datafile/o1_mf_system_fz01xnqo_.dbf
By looking at the SYSTEM tablespace (file 1) I can see exactly what is happening with the BCT file.
The first version marks the starting SCN prior to the first backup after creating the file.
The second version marks the checkpoint SCN of the first backup (low), and the SCN prior to the second backup(high).
These marks, LOW/HIGH SCN, can be used to identify the blocks that changed between the backups.
Now I am going to perform a few more incremental backups with a few changes to the USERS tablespace and see what happens to the SYSTEM and USERS tablespaces versions.
First here is my query.
col tablespace_name format a10
select (select tablespace_name from dba_data_files where file_id=fno) tablespace_name,vercnt,to_char(vertime,'mm/dd/yy hh24:mi:ss') vertime,low, high from x$krcfbh
where fno in (select file_id from dba_data_files)
order by 1,2;
Now let's see what what my backups look for File 1 (SYSTEM tablespace).
List of Backup Sets
===================
File LV Type Ckp SCN Ckp Time Abs Fuz SCN Sparse Name
---- -- ---- ---------- ----------------- ----------- ------ ----
1 0 Incr 8362002 07/29/19 09:46:59 NO /home/oracle/app/oracle/oradata/BSG/datafile/o1_mf_system_fz01xnqo_.dbf
1 1 Incr 8363962 07/29/19 10:04:00 NO /home/oracle/app/oracle/oradata/BSG/datafile/o1_mf_system_fz01xnqo_.dbf
1 1 Incr 8364371 07/29/19 10:13:21 NO /home/oracle/app/oracle/oradata/BSG/datafile/o1_mf_system_fz01xnqo_.dbf
1 1 Incr 8364482 07/29/19 10:15:19 NO /home/oracle/app/oracle/oradata/BSG/datafile/o1_mf_system_fz01xnqo_.dbf
1 1 Incr 8364637 07/29/19 10:16:38 NO /home/oracle/app/oracle/oradata/BSG/datafile/o1_mf_system_fz01xnqo_.dbf
1 1 Incr 8365075 07/29/19 10:26:23 NO /home/oracle/app/oracle/oradata/BSG/datafile/o1_mf_system_fz01xnqo_.dbf
1 1 Incr 8365373 07/29/19 10:29:13 NO /home/oracle/app/oracle/oradata/BSG/datafile/o1_mf_system_fz01xnqo_.dbf
1 1 Incr 8365578 07/29/19 10:30:21 NO /home/oracle/app/oracle/oradata/BSG/datafile/o1_mf_system_fz01xnqo_.dbf
1 1 Incr 8365763 07/29/19 10:30:58 NO /home/oracle/app/oracle/oradata/BSG/datafile/o1_mf_system_fz01xnqo_.dbf
Let's see what my backups look like for File 7 (USERS tablespace)
List of Backup Sets
===================
File LV Type Ckp SCN Ckp Time Abs Fuz SCN Sparse Name
---- -- ---- ---------- ----------------- ----------- ------ ----
7 0 Incr 8362002 07/29/19 09:46:59 NO /home/oracle/app/oracle/oradata/BSG/datafile/o1_mf_users_fz01zl49_.dbf
7 1 Incr 8363962 07/29/19 10:04:00 NO /home/oracle/app/oracle/oradata/BSG/datafile/o1_mf_users_fz01zl49_.dbf
7 1 Incr 8364371 07/29/19 10:13:21 NO /home/oracle/app/oracle/oradata/BSG/datafile/o1_mf_users_fz01zl49_.dbf
7 1 Incr 8364482 07/29/19 10:15:19 NO /home/oracle/app/oracle/oradata/BSG/datafile/o1_mf_users_fz01zl49_.dbf
7 1 Incr 8364637 07/29/19 10:16:38 NO /home/oracle/app/oracle/oradata/BSG/datafile/o1_mf_users_fz01zl49_.dbf
7 1 Incr 8365075 07/29/19 10:26:23 NO /home/oracle/app/oracle/oradata/BSG/datafile/o1_mf_users_fz01zl49_.dbf
7 1 Incr 8365373 07/29/19 10:29:13 NO /home/oracle/app/oracle/oradata/BSG/datafile/o1_mf_users_fz01zl49_.dbf
7 1 Incr 8365578 07/29/19 10:30:21 NO /home/oracle/app/oracle/oradata/BSG/datafile/o1_mf_users_fz01zl49_.dbf
7 1 Incr 8365763 07/29/19 10:30:58 NO /home/oracle/app/oracle/oradata/BSG/datafile/o1_mf_users_fz01zl49_.dbf
My backups for both of these match.
Notice that I performed an Incremental level 0 backup, and then 8 Incremental level 1 backups.
I actually performed both differential backups and cumulative backups but it didn't matter.
Now let's look at the block change tracking file for these 2 tablespaces (system and users)
TABLESPACE VERCNT VERTIME LOW HIGH
---------- ---------- ----------------- ---------- ----------
SYSTEM 3 07/29/19 10:04:00 8363961 8364370
SYSTEM 4 07/29/19 10:13:21 8364370 8364481
SYSTEM 5 07/29/19 10:15:19 8364481 8364636
SYSTEM 6 07/29/19 10:16:38 8364636 8365074
SYSTEM 7 07/29/19 10:26:23 8365074 8365372
SYSTEM 8 07/29/19 10:29:13 8365372 8365577
SYSTEM 9 07/29/19 10:30:20 8365577 8365762
SYSTEM 10 07/29/19 10:30:58 0 0
USERS 1 07/29/19 09:21:18 0 8362001
USERS 2 07/29/19 09:46:59 8362001 8364481
USERS 3 07/29/19 10:15:19 8364481 8364636
Very interesting.. Since I only made few changes to the users tablespace it has 3 versions, the oldest of which is the full backup.
The system tablespace has gone over 7 versions it no longer has the original version from the level 0 backup.
Now let's see if it used the BCT files for the backups.
FILE# Creation Time INCREMENTAL_LEVEL INCREMENTAL_CHANGE# CHECKPOINT_CHANGE# USED BCT
---------- ----------------- ----------------- ------------------- ------------------ ---
1 07/29/19 09:47:24 0 0 8362002 YES
1 07/29/19 10:04:02 1 8362002 8363962 YES
1 07/29/19 10:13:21 1 8363962 8364371 YES
1 07/29/19 10:15:19 1 8364371 8364482 YES
1 07/29/19 10:16:38 1 8364482 8364637 YES
1 07/29/19 10:26:24 1 8364637 8365075 YES
1 07/29/19 10:29:14 1 8362002 8365373 YES
1 07/29/19 10:30:22 1 8362002 8365578 YES
1 07/29/19 10:31:10 1 8362002 8365763NO
7 07/29/19 09:47:00 0 0 8362002 YES
7 07/29/19 10:04:02 1 8362002 8363962 YES
7 07/29/19 10:13:21 1 8364300 8364371 YES
7 07/29/19 10:15:19 1 8364371 8364482 YES
7 07/29/19 10:16:38 1 8364482 8364637 YES
7 07/29/19 10:26:24 1 8364637 8365075 YES
7 07/29/19 10:29:14 1 8362002 8365373 YES
7 07/29/19 10:30:22 1 8362002 8365578 YES
7 07/29/19 10:30:59 1 8362002 8365763 YES
WOW... Notice that the system tablespace (FILE #1) could not use the BCT file for the last backup, but the backup of the user tablespace (FILE #7) could because there were no changes between a few of the backups.
I can also see that the high SCN for users, 8364636,is older than the high SCN for system.
I am going to change users, and perform an incremental to see what happens to the LOW/HIGH scn in the next version.
TABLESPACE VERCNT VERTIME LOW HIGH
---------- ---------- ----------------- ---------- ----------
USERS 1 07/29/19 09:21:18 0 8362001
USERS 2 07/29/19 09:46:59 8362001 8364481
USERS 3 07/29/19 10:15:19 8364481 8364636
USERS 4 07/29/19 10:16:38 8364636 8366975
After the backup, there is no gap in the SCN numbers. It created a new version that contained the changes between the previous version HIGH and the SCN of the incremental backup.
I also performed a Level backup of the database, and since USERS didn't change, a new version wasn't created in the BCT for the USERS tablespace.
So what have we learned about the BCT ?
1) The BCT file keeps a bit map of changes between 2 SCN numbers
2) The BCT file keeps changes on a file level.
3) If a file didn't change between backups, it doesn't create a new record in the BCT file. It doesn't matter if the backup is an incremental or full backup.
4) By default 7 versions are kept for EACH DATAFILE. After 7 days of backups, some datafiles may still use the BCT if they haven't changed.
So how is the BCT file used ?
For full backups -
RMAN first determines if a BCT file is in use and is valid.
On a datafile basis, RMAN next determines if there are any blocks that have changed for that datafile since the last "version" record was stored.
-- If there were no changes recorded, then no version record is created
ELSE
-- If there is a single record with a LOW SCN and HIGH SCN of 0, then a record is created containing a LOW SCN of 0 and a HIGH SCN of the current SCN. This is the starting record.
ELSE
-- A new version record is created containing a LOW SCN value of the previous version record +1, and a HIGH SCN of the current checkpoint SCN number prior to the full backup.
For Incremental Cumulative backups -
RMAN first determines if a BCT file is in use and is valid.
On a datafile basis, RMAN next determines if there are any blocks that have changed for that datafile since the last "version" was stored.
-- If there were no changes recorded, then no version record is created
ELSE
-- If there is a single record with a LOW SCN and HIGH SCN of 0, then a record is created containing a LOW SCN of 0 and a HIGH SCN of the current SCN. This is the starting record.
ELSE
-- If there changes to the datafile, a new version record is created containing a LOW SCN value of the previous version record, and a HIGH SCN of the SCN number prior to the full backup.
RMAN then determines if it can use the BCT information.
RMAN will determine the checkpoint SCN of the last incremental level 0 backup for this datafile.
It will then check the BCT
- If the checkpoint is after the HIGH SCN of the most current version, then no blocks changed.
ELSE
-- If the checkpoint falls with a version record that has LOW SCN of 0, then the BCT file is too new and can't be used
ELSE
-- If the checkpoint does not fall with a version record then the there have been too many versions and the BCT file can't be used.
ELSE
-- Find the Version record whose LOW SCN matches the checkpoint of the last full backup, and find all version records created since that point. Combine the bitmaps for all the versions and this will contain the blocks that changed.
For Differential Cumulative backups -
RMAN first determines if a BCT file is in use and is valid.
On a datafile basis, RMAN next determines if there are any blocks that have changed for that datafile since the last "version" was stored.
-- If there were no changes recorded, then no version record is created
ELSE
-- If there is a single record with a LOW SCN and HIGH SCN of 0, then a record is created containing a LOW SCN of 0 and a HIGH SCN of the current SCN. This is the starting record.
ELSE
-- If there changes to the datafile, a new version record is created containing a LOW SCN value of the previous version record, and a HIGH SCN of the SCN number prior to the full backup.
RMAN then determines if it can use the BCT information.
RMAN will determine the checkpoint SCN of the last incremental level 0 OR the last incremental level 1 backup for this datafile.
It will then check the BCT
- If the checkpoint is after the HIGH SCN of the most current version, then no blocks changed.
ELSE
-- If the current version record that has LOW SCN of 0, then the BCT file is too new and can't be used
ELSE
-- Use the last version record to find the changed blocks
NOTE : if using multiple backups strategies, and TAGS to identify backups, this will complicate the process.
ZDLRA PROCESS
For the ZDLRA, the process is very simple. As you go through the steps, you see that that new version record is create for each datafile that has any changes since the last backup (regardless of the backup type).
When an incremental cumulative backup is executed and sent to the ZDLRA, it compares the checkpoint scn of the last full backup in the catalog (the virtual full).
Because it is using the RMAN catalog, the RMAN client always compares to the last virtual full.
Keeping 7 days of BCT history works fine because it always compares to the previous backup.