There are situations when we lose redo log files which are ACTIVE.
Let us see the scenario how to recover the database when we come to that
situation.
Scenario 1 :
Ø
Redo log files are ACTIVE and ARCHIVED.
Ø
CHECKPOINT is completed.
Ø
No need to restore from backup.
1. Let us see the redo log files status
SQL> select
a.group#,a.archived,a.thread#,b.member,a.status from v$log a, v$logfile b where
a.group#=b.group# order by a.group#;
GROUP# ARC THREAD# MEMBER
STATUS
---------- --- ---------- -----------------------------------------------------------------
----------------
1 YES 1
/data1/noasmdb/redofiles/redo_g01a.log INACTIVE
1 YES 1
/data1/noasmdb/redofiles/redo_g01b.log INACTIVE
2 YES 1
/data1/noasmdb/redofiles/redo_g02b.log ACTIVE
2 YES 1
/data1/noasmdb/redofiles/redo_g02a.log ACTIVE
3 NO 1 /data1/noasmdb/redofiles/redo_g03b.log CURRENT
3 NO 1
/data1/noasmdb/redofiles/redo_g03a.log CURRENT
2. In the above o/p, group# 2 is ACTIVE and
ARCHIVED. Now, we will create some
tables and perform inserts, and then we will do a redolog switch. Let’s create table and insert records to store
in Group 3
declare
begin
execute immediate 'create table test1.test_active4 (a
number)';
for i in 1 .. 100
loop
execute immediate 'insert into test1.test_active4 values
(:a)' using i;
end loop;
commit;
end;
/
SQL> alter system switch logfile;
System altered.
SQL> select
a.group#,a.archived,a.thread#,b.member,a.status from v$log a, v$logfile b where
a.group#=b.group# order by a.group#;
GROUP# ARC THREAD# MEMBER
STATUS
---------- --- ----------
-----------------------------------------------------------------
----------------
1 NO 1
/data1/noasmdb/redofiles/redo_g01a.log CURRENT
1 NO 1
/data1/noasmdb/redofiles/redo_g01b.log CURRENT
2 YES 1
/data1/noasmdb/redofiles/redo_g02b.log INACTIVE
2 YES 1
/data1/noasmdb/redofiles/redo_g02a.log INACTIVE
3 YES 1
/data1/noasmdb/redofiles/redo_g03b.log ACTIVE
3 YES 1 /data1/noasmdb/redofiles/redo_g03a.log ACTIVE
3. In the above result, we see, group# 3 is ACTIVE and
ARCHIVED. Now, we will create a table and insert recrords for group# 1.
SQL> create table test1.test_active5 (a number);
Table created.
declare
begin
for i in 1 .. 100
loop
execute immediate 'insert into test1.test_active5 values
(:a)' using i;
end loop;
commit;
end;
/
SQL> alter system switch logfile;
System altered.
SQL> select a.group#,a.archived,a.thread#,b.member,a.status
from v$log a, v$logfile b where a.group#=b.group# order by a.group#;
GROUP# ARC THREAD# MEMBER
STATUS
---------- --- ----------
------------------------------------------------------------ ----------------
1 YES 1
/data1/noasmdb/redofiles/redo_g01a.log ACTIVE
1 YES 1
/data1/noasmdb/redofiles/redo_g01b.log ACTIVE
2 NO 1 /data1/noasmdb/redofiles/redo_g02b.log CURRENT
2 NO 1
/data1/noasmdb/redofiles/redo_g02a.log CURRENT
3 YES 1
/data1/noasmdb/redofiles/redo_g03b.log ACTIVE
3 YES
1
/data1/noasmdb/redofiles/redo_g03a.log ACTIVE
6 rows selected.
In the above result, now redolog group# 1 and 3 are in
ACTIVE and ARCHIVED state.
4. Now, again , we will create another table for having data
in redo log 2 group.
SQL> create table test1.test_active6 (a number);
Table created.
SQL> declare
begin
for i in 1 .. 100
loop
execute immediate 'insert into test1.test_active6 values
(:a)' using i;
end loop;
commit;
end;
/
PL/SQL procedure successfully completed.
SQL> alter system switch logfile;
System altered.
SQL>
SQL> select
a.group#,a.archived,a.thread#,b.member,a.status from v$log a, v$logfile b where
a.group#=b.group# order by a.group#;
GROUP# ARC THREAD# MEMBER
STATUS
---------- --- ----------
------------------------------------------------------------ ----------------
1 YES 1 /data1/noasmdb/redofiles/redo_g01a.log ACTIVE
1 YES 1
/data1/noasmdb/redofiles/redo_g01b.log ACTIVE
2 YES 1
/data1/noasmdb/redofiles/redo_g02b.log ACTIVE
2 YES 1 /data1/noasmdb/redofiles/redo_g02a.log ACTIVE
3 NO 1
/data1/noasmdb/redofiles/redo_g03b.log CURRENT
3 NO 1
/data1/noasmdb/redofiles/redo_g03a.log CURRENT
6 rows selected.
5. Finally , redo log group 3 is in CURRENT state and let us take the count of all 3 tables.
5. Finally , redo log group 3 is in CURRENT state and let us take the count of all 3 tables.
SQL> select count(1) from test_active4;
COUNT(1)
----------
100
SQL> select count(1) from test_active5;
COUNT(1)
----------
100
SQL> select count(1) from test_active6;
COUNT(1)
----------
100
SQL> select
a,b,c from (select count(1) a from
test_active4),(select count(1) b from test_active5),(select count(1) c from test_active6);
A B
C
---------- ---------- ----------
100 100 100
6. Now, we will remove/move the redo log files of group 2 which are ACTIVE and ARCHIVED.
6. Now, we will remove/move the redo log files of group 2 which are ACTIVE and ARCHIVED.
[oracle@vm1 redofiles]$ rm redo_g02b.log
[oracle@vm1 redofiles]$ rm redo_g02a.log
[oracle@vm1 redofiles]$ ls -ltrh
total 401M
drwxr-x--- 5 oracle asmadmin 4.0K Feb 8 13:09 NOASMDB
-rw-r----- 1 oracle asmadmin 101M Feb 14 11:21
redo_g01b.log
-rw-r----- 1 oracle asmadmin 101M Feb 14 11:21
redo_g01a.log
-rw-r----- 1 oracle asmadmin 101M Feb 14 11:32
redo_g03b.log
-rw-r----- 1 oracle asmadmin 101M Feb 14 11:32
redo_g03a.log
[oracle@vm1 redofiles]$
7. If the database is still running and the lost active redo log is not the current log, then issue the ALTER SYSTEM CHECKPOINT statement. If successful, then the active redo log becomes inactive.
7. If the database is still running and the lost active redo log is not the current log, then issue the ALTER SYSTEM CHECKPOINT statement. If successful, then the active redo log becomes inactive.
SQL> select
a.group#,a.archived,a.thread#,b.member,a.status from v$log a, v$logfile b where
a.group#=b.group# order by a.group#;
GROUP# ARC THREAD# MEMBER STATUS
---------- --- ----------
-----------------------------------------------------------------
----------------
1 YES 1
/data1/noasmdb/redofiles/redo_g01a.log INACTIVE
1 YES 1
/data1/noasmdb/redofiles/redo_g01b.log INACTIVE
2 YES 1
/data1/noasmdb/redofiles/redo_g02b.log INACTIVE
2 YES 1
/data1/noasmdb/redofiles/redo_g02a.log
INACTIVE
3 NO 1
/data1/noasmdb/redofiles/redo_g03b.log CURRENT
3 NO 1
/data1/noasmdb/redofiles/redo_g03a.log CURRENT
In our case, CHECKPOINT got completed automatically by
oracle and it went to INACTIVE state. So, if you give “alter system checkpoint”
again, then it will be successful without any errors.
SQL> alter system checkpoint;
System altered.
SQL> select a.group#,a.archived,a.thread#,b.member,a.status
from v$log a, v$logfile b where a.group#=b.group# order by a.group#;
GROUP# ARC THREAD# MEMBER
STATUS
---------- --- ---------- -----------------------------------------------------------------
----------------
1 YES 1
/data1/noasmdb/redofiles/redo_g01a.log INACTIVE
1 YES 1
/data1/noasmdb/redofiles/redo_g01b.log
INACTIVE
2 YES 1
/data1/noasmdb/redofiles/redo_g02b.log INACTIVE
2 YES 1
/data1/noasmdb/redofiles/redo_g02a.log INACTIVE
3 NO 1
/data1/noasmdb/redofiles/redo_g03b.log CURRENT
3 NO 1
/data1/noasmdb/redofiles/redo_g03a.log CURRENT
6 rows selected.
Now we are having a redo log 2 in INACTIVE and ARCHIVED state
8. Also our database is available. No need to shutdown, in this case, we have issue the following command to recover the redo log file.
8. Also our database is available. No need to shutdown, in this case, we have issue the following command to recover the redo log file.
SQL> alter database clear
logfile group 2;
Database altered.
SQL> select
a.group#,a.archived,a.thread#,b.member,a.status from v$log a, v$logfile b where
a.group#=b.group# order by a.group#;
GROUP# ARC THREAD# MEMBER
STATUS
---------- --- ----------
-----------------------------------------------------------------
----------------
1 YES 1
/data1/noasmdb/redofiles/redo_g01a.log INACTIVE
1 YES 1
/data1/noasmdb/redofiles/redo_g01b.log INACTIVE
2 YES 1
/data1/noasmdb/redofiles/redo_g02b.log UNUSED
2 YES 1
/data1/noasmdb/redofiles/redo_g02a.log UNUSED
3 NO 1
/data1/noasmdb/redofiles/redo_g03b.log
CURRENT
3 NO 1
/data1/noasmdb/redofiles/redo_g03a.log CURRENT
6 rows selected.
9. Now, we have cleared the redo log for group#2. Let’s see whether we have data for our tables .Ideally, we created and inserted data for test_active6 for redo log group 2.
9. Now, we have cleared the redo log for group#2. Let’s see whether we have data for our tables .Ideally, we created and inserted data for test_active6 for redo log group 2.
SQL> select count(1) from test1.test_active6;
COUNT(1)
----------
100
Here we have records, it was because, those redo log
files were ARCHIVED and became
INACTIVE. Hence the transaction was NOT LOST.
Scenario 2: Incomplete recovery/Media Recovery
Ø
Redo log files are ACTIVE and ARCHIVED.
Ø
CHECKPOINT is NOT completed.
Ø
Need to restore from backup until FIRST_CHANGE#
SCN of the missing redo log group#
Since we do not lose transaction due to CHECPOINT completion,
we will simulate the crash by killing the instance and avoid checkpoint.
1.
For this step, first we will identify the
process id of smon.
[oracle@vm1 redofiles]$ ps -ef
|grep smon
oracle 7678
1 0 11:59 ? 00:00:01 ora_smon_noasmdb
oracle 8079
7229 0 12:11 pts/6 00:00:00 grep smon
2. Then
in three different UNIX terminals, we will prepare the following commands (each
on one terminal). In this way with just one RETURN button I'm able to remove
the ACTIVE redo log group and kill the instance, without any checkpoint and
"perhaps" loosing transactions. Do
not execute the commands now.
mv redo_g01b.log redo_g01b.log.old; mv redo_g01a.log
redo_g01a.log.old; kill -9 7678
mv redo_g02b.log redo_g02b.log.old; mv redo_g02a.log
redo_g02a.log.old; kill -9 7678
mv redo_g03b.log redo_g03b.log.old; mv redo_g03a.log
redo_g03a.log.old; kill -9 7678
3.
Now, we will create tables and perform switching
of logfiles after every table creation.
SQL> select
a.group#,a.archived,a.thread#,b.member,a.status from v$log a, v$logfile b where
a.group#=b.group# order by a.group#;
GROUP# ARC THREAD# MEMBER
STATUS
---------- --- ---------- -----------------------------------------------------------------
----------------
1 YES 1
/data1/noasmdb/redofiles/redo_g01a.log INACTIVE
1 YES 1
/data1/noasmdb/redofiles/redo_g01b.log INACTIVE
2 YES 1
/data1/noasmdb/redofiles/redo_g02b.log INACTIVE
2 YES 1
/data1/noasmdb/redofiles/redo_g02a.log INACTIVE
3 NO 1 /data1/noasmdb/redofiles/redo_g03b.log CURRENT
3 NO 1
/data1/noasmdb/redofiles/redo_g03a.log CURRENT
Next we will create another table for storing in Redo log group #3.
declare
begin
execute immediate 'create table test1.test_active4 (a
number)';
for i in 1 .. 100
loop
execute immediate 'insert into test1.test_active4 values
(:a)' using i;
end loop;
commit;
end;
/
SQL> alter system switch logfile;
System altered.
SQL> select
a.group#,a.archived,a.thread#,b.member,a.status from v$log a, v$logfile b where
a.group#=b.group# order by a.group#;
GROUP# ARC THREAD# MEMBER
STATUS
---------- --- ----------
----------------------------------------------------------------- ----------------
1 YES 1
/data1/noasmdb/redofiles/redo_g01a.log INACTIVE
1 YES 1
/data1/noasmdb/redofiles/redo_g01b.log INACTIVE
2 NO 1 /data1/noasmdb/redofiles/redo_g02b.log CURRENT
2 NO 1
/data1/noasmdb/redofiles/redo_g02a.log CURRENT
3 YES 1
/data1/noasmdb/redofiles/redo_g03b.log ACTIVE
3 YES 1
/data1/noasmdb/redofiles/redo_g03a.log ACTIVE
Next we will create another table for storing in Redo log
group #2.
SQL> create table test1.test_active5 (a number);
Table created.
declare
begin
for i in 1 .. 100
loop
execute immediate 'insert into test1.test_active5 values
(:a)' using i;
end loop;
commit;
end;
/
SQL> alter
system switch logfile;
System altered.
SQL> select
a.group#,a.archived,a.thread#,b.member,a.status from v$log a, v$logfile b where
a.group#=b.group# order by a.group#;
GROUP# ARC THREAD# MEMBER
STATUS
---------- --- ----------
-----------------------------------------------------------------
----------------
1 NO 1 /data1/noasmdb/redofiles/redo_g01a.log CURRENT
1 NO 1
/data1/noasmdb/redofiles/redo_g01b.log CURRENT
2 YES 1
/data1/noasmdb/redofiles/redo_g02b.log ACTIVE
2 YES 1
/data1/noasmdb/redofiles/redo_g02a.log ACTIVE
3 YES 1
/data1/noasmdb/redofiles/redo_g03b.log ACTIVE
3 YES 1 /data1/noasmdb/redofiles/redo_g03a.log ACTIVE
Next, we are going to create a table for storing the data
in Redo GROUP# 1
SQL> create table test1.test_active6 (a number);
Table created.
SQL> declare
begin
for i in 1 .. 100
loop
execute immediate 'insert into test1.test_active6 values
(:a)' using i;
end loop;
commit;
end;
/
SQL> alter system switch logfile;
System altered.
SQL> select
a.group#,a.archived,a.thread#,b.member,a.status from v$log a, v$logfile b where
a.group#=b.group# order by a.group#;
GROUP# ARC THREAD# MEMBER
STATUS
---------- --- ----------
-----------------------------------------------------------------
----------------
1 YES 1 /data1/noasmdb/redofiles/redo_g01a.log ACTIVE
1 YES 1
/data1/noasmdb/redofiles/redo_g01b.log ACTIVE
2 YES 1
/data1/noasmdb/redofiles/redo_g02b.log ACTIVE
2 YES 1
/data1/noasmdb/redofiles/redo_g02a.log ACTIVE
3 NO 1
/data1/noasmdb/redofiles/redo_g03b.log CURRENT
3 NO 1 /data1/noasmdb/redofiles/redo_g03a.log CURRENT
Now, we see two groups are in ACTIVE and ARCHIVED status.
4.
Now, we will remove all files for redo log
group# 1 as mentioned in step 2.
[oracle@vm1 redofiles]$ mv redo_g01b.log
redo_g01b.log.old; mv redo_g01a.log redo_g01a.log.old; kill -9 7678
[oracle@vm1 redofiles]$ ls -ltrh
total 601M
drwxr-x--- 5 oracle asmadmin 4.0K Feb 8 13:09 NOASMDB
-rw-r----- 1 oracle asmadmin 101M Feb 14 12:07
redo_g02b.log
-rw-r----- 1 oracle asmadmin 101M Feb 14 12:07
redo_g02a.log
-rw-r----- 1 oracle asmadmin 101M Feb 14 12:09
redo_g01b.log.old
-rw-r----- 1 oracle asmadmin 101M Feb 14 12:09
redo_g01a.log.old
-rw-r----- 1 oracle asmadmin 101M Feb 14 12:12
redo_g03b.log
-rw-r----- 1 oracle asmadmin 101M Feb 14 12:12 redo_g03a.log
5.
Next, we will start the instance in MOUNT and
query v$log and v$logfile to see the status of the log files. Redo logs are still in the same status.
SQL> select
a.group#,a.archived,a.thread#,b.member,a.status from v$log a, v$logfile b where
a.group#=b.group# order by a.group#;
GROUP# ARC THREAD# MEMBER
STATUS
---------- --- ----------
-----------------------------------------------------------------
----------------
1 YES
1 /data1/noasmdb/redofiles/redo_g01a.log ACTIVE
1 YES 1
/data1/noasmdb/redofiles/redo_g01b.log ACTIVE
2 YES 1
/data1/noasmdb/redofiles/redo_g02b.log ACTIVE
2 YES 1
/data1/noasmdb/redofiles/redo_g02a.log ACTIVE
3 NO 1
/data1/noasmdb/redofiles/redo_g03b.log CURRENT
3 NO 1
/data1/noasmdb/redofiles/redo_g03a.log CURRENT
6.
Now, we will give “alter system checkpoint” and
see what happens.
SQL> alter system checkpoint;
alter system checkpoint
*
ERROR at line 1:
ORA-01109: database not open
7.
Now, we will try to open the db , it doesn’t
open.
SQL> alter database open;
alter database open
*
ERROR at line 1:
ORA-00313: open failed for members of log group 1 of
thread 1
ORA-00312: online log 1 thread 1:
'/data1/noasmdb/redofiles/redo_g01b.log'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
ORA-00312: online log 1 thread 1:
'/data1/noasmdb/redofiles/redo_g01a.log'
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3
8.
Oracle documentation says, we need to recover
the database until the FIRST_CHANGE# value SCN of the missing redo log files.
SQL> select
group#,sequence#,members,archived,status,first_change#,next_change# from v$log
order by group#;
GROUP# SEQUENCE#
MEMBERS ARC STATUS
FIRST_CHANGE# NEXT_CHANGE#
---------- ---------- ---------- --- ----------------
------------- ------------
1 11 2 YES ACTIVE 1365418 1365494
2 10 2 YES ACTIVE 1365349 1365418
3 12 2 NO
CURRENT 1365494 2.8147E+14
Here missing redo log group is Group# 1 .
9.
Now, connect to rman.
[oracle@vm1 backup]$ rman target
sys/sys catalog=rman_cat/rman_cat@testdb
Recovery Manager: Release
11.2.0.3.0 - Production on Fri Feb 14 12:18:05 2014
Copyright (c) 1982, 2011, Oracle
and/or its affiliates. All rights
reserved.
connected to target database:
NOASMDB (DBID=1704132256, not open)
connected to recovery catalog
database
RMAN> restore database until
scn 1365418;
Starting restore at 14-FEB-14
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=135
device type=DISK
flashing back control file to
SCN 1365418
channel ORA_DISK_1: starting
datafile backup set restore
channel ORA_DISK_1: specifying
datafile(s) to restore from backup set
channel ORA_DISK_1: restoring
datafile 00001 to /data1/noasmdb/datafile/system01.dbf
channel ORA_DISK_1: restoring
datafile 00002 to /data1/noasmdb/datafile/sysaux01.dbf
channel ORA_DISK_1: restoring
datafile 00003 to /data1/noasmdb/datafile/testbkp01.dbf
channel ORA_DISK_1: restoring
datafile 00004 to /data1/noasmdb/datafile/users01.dbf
channel ORA_DISK_1: restoring
datafile 00005 to /data1/noasmdb/datafile/undotbs01.dbf
channel ORA_DISK_1: restoring
datafile 00006 to /data1/noasmdb/datafile/newtest01.dbf
channel ORA_DISK_1: restoring
datafile 00007 to /data1/noasmdb/datafile/novalidtbs01.dbf
channel ORA_DISK_1: reading from
backup piece /home/oracle/backup/FullDBNOASM_i6p0ji78_1_1
channel ORA_DISK_1: piece
handle=/home/oracle/backup/FullDBNOASM_i6p0ji78_1_1 tag=FULL DB
channel ORA_DISK_1: restored
backup piece 1
channel ORA_DISK_1: restore
complete, elapsed time: 00:12:16
Finished restore at 14-FEB-14
RMAN> recover database until
scn 1365418;
Starting recover at 14-FEB-14
using channel ORA_DISK_1
starting media recovery
archived log for thread 1 with
sequence 2 is already on disk as file
/data1/noasmdb/archivelog/1_2_839349663.dbf
archived log for thread 1 with
sequence 3 is already on disk as file
/data1/noasmdb/archivelog/1_3_839349663.dbf
archived log for thread 1 with
sequence 4 is already on disk as file
/data1/noasmdb/archivelog/1_4_839349663.dbf
archived log for thread 1 with
sequence 5 is already on disk as file
/data1/noasmdb/archivelog/1_5_839349663.dbf
archived log for thread 1 with
sequence 6 is already on disk as file
/data1/noasmdb/archivelog/1_6_839349663.dbf
archived log for thread 1 with
sequence 7 is already on disk as file
/data1/noasmdb/archivelog/1_7_839349663.dbf
archived log for thread 1 with
sequence 8 is already on disk as file /data1/noasmdb/archivelog/1_8_839349663.dbf
archived log for thread 1 with
sequence 9 is already on disk as file
/data1/noasmdb/archivelog/1_9_839349663.dbf
archived log for thread 1 with
sequence 10 is already on disk as file /data1/noasmdb/redofiles/redo_g02a.log
archived log file name=/data1/noasmdb/archivelog/1_2_839349663.dbf
thread=1 sequence=2
archived log file
name=/data1/noasmdb/archivelog/1_3_839349663.dbf thread=1 sequence=3
archived log file
name=/data1/noasmdb/archivelog/1_4_839349663.dbf thread=1 sequence=4
archived log file
name=/data1/noasmdb/archivelog/1_5_839349663.dbf thread=1 sequence=5
archived log file
name=/data1/noasmdb/archivelog/1_6_839349663.dbf thread=1 sequence=6
archived log file
name=/data1/noasmdb/archivelog/1_7_839349663.dbf thread=1 sequence=7
archived log file
name=/data1/noasmdb/archivelog/1_8_839349663.dbf thread=1 sequence=8
archived log file
name=/data1/noasmdb/archivelog/1_9_839349663.dbf thread=1 sequence=9
media recovery complete, elapsed
time: 00:00:20
Finished recover at 14-FEB-14
RMAN> alter database open
resetlogs;
database opened
new incarnation of database
registered in recovery catalog
starting full resync of recovery
catalog
full resync complete
10.
Even the redo log files for group 1 are created
by RMAN recover.
[oracle@vm1 redofiles]$ ls -ltrh
total 801M
drwxr-x--- 5 oracle asmadmin
4.0K Feb 8 13:09 NOASMDB
-rw-r----- 1 oracle asmadmin
101M Feb 14 12:09 redo_g01b.log.old
-rw-r----- 1 oracle asmadmin
101M Feb 14 12:09 redo_g01a.log.old
-rw-r----- 1 oracle asmadmin
101M Feb 14 12:32 redo_g03b.log
-rw-r----- 1 oracle asmadmin
101M Feb 14 12:32 redo_g03a.log
-rw-r----- 1 oracle asmadmin
101M Feb 14 12:32 redo_g02b.log
-rw-r----- 1 oracle asmadmin
101M Feb 14 12:32 redo_g02a.log
-rw-r----- 1 oracle asmadmin 101M Feb 14 12:33 redo_g01b.log
-rw-r----- 1 oracle asmadmin 101M Feb 14 12:33 redo_g01a.log
11.
Next, we need to see whether TEST_ACTIVE6 table exists or not exists. For us, it should not exist as redo log file
1 was removed.
Let see..
SQL> select count(1) from
test_active6;
select count(1) from
test_active6
*
ERROR at line 1:
ORA-00942: table or view does
not exist
The table does NOT exists which
means we lose few transactions and did Incomplete Recovery.
Conclusion:
When we miss ACTIVE/CURRENT redo
log file group which is ARCHIVED, we need to perfrom INCOMPLETE/MEDIA RECOVERY
from the backup.
No comments:
Post a Comment