|
Larry,
I've got 3 ProtecTIERS and have 2 more on order. We have them located in
different states around the US and we use the dedup and replication
features to copy the virtual tapes across the wire. I tried SPHiNX back
before it was really ready for prime time, but based on the limits of
DEDUP, I would say Protectier is far superior technology. It also runs
circles around Data domain.
We are getting 48:1 Dedup ratio and can transmit 2TB in about 3 hours
because it only sends the delta across the wire.
Gavin.
Stryker Corporation
On 8/17/2015 3:32 PM, DrFranken wrote:
I work a lot with the Crossroads SPHiNX VTLs. I have been very pleased--
with the functionality and performance they offer and they truly do appear
as native tape so IPLing from them and full system restores work
perfectly. Currently they emulate up to LTO5 drives but since the tapes
are really virtual files you can put an LTO1 'tape' into an LTO5 drive and
it works, and you can also put an LTO5 'tape' into an LTO1 drive and that
works too!!
The one area they do not have is dedup. They have engineering working on
it so I'm told but so far they rely purely on compression instead which I
would say averages around 3 to 1 most of the time.
The way they store the files though is simple and that means moving them,
off-site copies, and replication of them simple and easy. It's just not so
quick because you are copying all the data all the time.
One of the issues no matter what method you use though is that IBM i
still needs to read the file and send it out the SCSI/fiber/SAS port to the
VTL. Until it gets to the VTL it's just a backup stream from IBM i so if
you are arm limited on IBM i or speed limited on the port all that
deduplication just means you'll be sending billions and billions of bytes
to their demise! Sure you don't have to wait for it to write to tape or
disk but you still gotta read it and send it.
I can see the little man in the VTL: "DUP toss it. DUP toss it. DUP toss
it DUP toss it....." :-)
- Larry "DrFranken" Bolhuis
www.Frankeni.com
www.iDevCloud.com - Personal Development IBM i timeshare service.
www.iInTheCloud.com - Commercial IBM i Cloud Hosting.
On 8/17/2015 3:22 PM, Steinmetz, Paul wrote:
Larry,
I'm looking at the dedup, wondering how much would be saved (both space
and time) in large history files, new data added maybe weekly or monthly.
However, I've also read a VTL solution is not necessarily faster than
tape.
I'm sure it depends which tape one is comparing.
Paul
-----Original Message-----
From: MIDRANGE-L [mailto:midrange-l-bounces@xxxxxxxxxxxx] On Behalf Of
DrFranken
Sent: Monday, August 17, 2015 2:48 PM
To: Midrange Systems Technical Discussion
Subject: Re: VTL and ProtecTIER deduplication
I had met Nancy, very nice, very smart. Sad the losses IBM (and by
extension WE) have endured....
This basically confirms what I'm thinking. But there must be some things
under the covers about the way that IBM i writes backups that causes this
process to work as well as the charts depict. 15 to 1 and up is pretty
doggone good! I can certainly see if I am backing up a library that
changes infrequently that dedup would be a great fit there but often
administrators already know this and don't take the time to back those
libraries up frequently as a result.
I am working with a customer right now that has 10s of millions of
documents in the IFS that are written once, changed/deleted never, and read
maybe. The collection is updated only once per month so it's backed up
immediately after that update, perfectly logical. Dedup would STILL help
here as the 10s of millions that are there this month will be there next
month and only the new ones this month should in theory cause new blocks of
data to be written.
Once question though, if all your backup copies are in fact just one
copy after deduplication it would be bad to have a crash on the device now
wouldn't it? :-)
- Larry "DrFranken" Bolhuis
www.Frankeni.com
www.iDevCloud.com - Personal Development IBM i timeshare service.
www.iInTheCloud.com - Commercial IBM i Cloud Hosting.
On 8/17/2015 2:23 PM, Kendall Kinnear wrote:
Larry, I don't know if these will help or not but a friend did locate
them for me in TechDocs. There was a great lady in Toronto by the name ofThis is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing
Nancy Roper who supported tape and specifically VTL. IBM laid her off a
couple of years ago unfortunately. Anyway, these are from her. Take a look
and see if they will help you at all.
http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/PRS4737
http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/PRS5075
Respectfully,
Kendall Kinnear
System Analyst
Standard Motor Products, Inc.
Work: 972-316-8169
Mobile: 940-293-7541
-----Original Message-----
From: MIDRANGE-L [mailto:midrange-l-bounces@xxxxxxxxxxxx] On Behalf Of
Kendall Kinnear
Sent: Monday, August 17, 2015 11:27 AM
To: Midrange Systems Technical Discussion <midrange-l@xxxxxxxxxxxx>
Subject: RE: VTL and ProtecTIER deduplication
I understand, it sounds like a lot of smoke and mirrors and I, like
you, typically want to understand the process more deeply than that.
I had a really good technical contact in ProtecTIER land last year but
he got laid off shortly before I left the BP business. Let me check with a
couple of people and see if there is a good technical discussion of de-dup
laying around.
Respectfully,
Kendall Kinnear
System Analyst
Standard Motor Products, Inc.
Work: 972-316-8169
Mobile: 940-293-7541
-----Original Message-----
From: MIDRANGE-L [mailto:midrange-l-bounces@xxxxxxxxxxxx] On Behalf Of
DrFranken
Sent: Monday, August 17, 2015 11:07 AM
To: Midrange Systems Technical Discussion <midrange-l@xxxxxxxxxxxx>
Subject: Re: VTL and ProtecTIER deduplication
So here's the thing about de-duplication.
Lets say the device is empty and this is my first backup. The first
block of data then by definition is unique so it can't be deduped. It gets
written. So now here comes block two, it needs to be compared. Not a huge
deal to compare one block to another but clearly they aren't going to
compare the entire block it must be a hash function of some sort. So what
is a block then? Is it just every 1KB? 16KB? 256KB? (the size of an LTO
Block) Every 1MB?
Then it gets harder because after there have been tens of thousands of
blocks written we can't compare to every one of them on the fly I wouldn't
expect, at least not without a hash anyway. And of course you gotta keep
track of how many tapes use a particular block or you could delete it when
you think it's no longer needed and that would be bad. So that tracking is
also part of the process. The thing is to me how do a significant number
of those blocks stay the same? If I add a few bytes to a text string in the
early part of a large table wouldn't that offset the bytes in every
subsequent block for that table, even that library, causing them to be
different almost every time? I don't know that to be true, I'm just
guessing.
I realize that you probably don't have the de-dup code handy to refer
to as I suspect that's kinda proprietary like but it IS an interesting
problem to me.
- Larry "DrFranken" Bolhuis
www.Frankeni.com
www.iDevCloud.com - Personal Development IBM i timeshare service.
www.iInTheCloud.com - Commercial IBM i Cloud Hosting.
On 8/17/2015 10:24 AM, Kendall Kinnear wrote:
You're going to really work my memory now. I haven't thought about
this stuff in over a year. :-)This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing
If I remember correctly, the deduplication is inline (on the fly) as
the data passes through the device and before it is written to disk. If you
think about it, it doesn't matter if all the data is available. The data
being processed is checked against what's stored on the disk and only the
dedup pointers are written if that is all is needed.
Yes you can setup the replication to happen automatically with no
human intervention.
I am pretty sure you can flag a virtual volume as read only but I
don't remember for sure.
Respectfully,
Kendall Kinnear
System Analyst
Standard Motor Products, Inc.
Work: 972-316-8169
Mobile: 940-293-7541
-----Original Message-----
From: MIDRANGE-L [mailto:midrange-l-bounces@xxxxxxxxxxxx] On Behalf
Of DrFranken
Sent: Monday, August 17, 2015 9:13 AM
To: Midrange Systems Technical Discussion <midrange-l@xxxxxxxxxxxx>
Subject: Re: VTL and ProtecTIER deduplication
Kendall,
Clearly you have very detailed knowledge of this environment! So
a few follow on questions if I may.
Does the de-duplication happen on the fly, that is during the save
or is that analysis and de-dup done after the save when all the data is
available?
Can you replicate a backup to a second ProtecTIER device so that
it gets off-site with no manual handling?
Can you flag a tape in the ProtecTIER VTL as 'read only'?
Thanks!
- Larry "DrFranken" Bolhuis
www.Frankeni.com
www.iDevCloud.com - Personal Development IBM i timeshare service.
www.iInTheCloud.com - Commercial IBM i Cloud Hosting.
On 8/14/2015 4:26 PM, Kendall Kinnear wrote:
1) I had a couple of clients using protecTIER when I was with a
Business Partner. They had vastly different deduplication ratios due toThis is the Midrange Systems Technical Discussion (MIDRANGE-L)
their data makeup.
2) The IBM Business Partners have access to a spreadsheet that you
can use with BRMS (and appropriate version/release/PTF) to estimate the
amount of physical storage required.
3) Any LTO media should be able to be migrated into the VTL using
DUPTAP functions.
4) You might want to change some things in BRMS. At a minimum the
tape library you default to. More thoughts on BRMS in later questions. The
thing to remember about the ProtecTIER is that the IBM i simply sees it as
a LTO tape library, nothing more and nothing less.
5) You'll need as many fibre adapters as you want permanent
connections in your partitions. If you don't mind moving adapters between
partitions then you can get by with fewer adapters.
6) Yes, you can run a large number of concurrent saves from different
partitions or the same partition, it depends on your configuration.
7) Speed, there's the rub. Last time I checked the ProtecTIER ran at
approximately LTO3 speeds (that was mid 2014). You make up for that by
having multiple virtual drives within the virtual library. Depending on the
workload of your backup from a partition, you may need a single fibre that
sees 3 or 4 drives assigned to that partition. There is not a 1 to 1 drive
to fibre ratio with a VTL, each connection can have multiple virtual
drives. Here's where you'd change BRMS to do parallel saves across multiple
virtual drives.
Respectfully,
Kendall Kinnear
System Analyst
Standard Motor Products, Inc.
Work: 972-316-8169
Mobile: 940-293-7541
-----Original Message-----
From: MIDRANGE-L [mailto:midrange-l-bounces@xxxxxxxxxxxx] On Behalf
Of Steinmetz, Paul
Sent: Friday, August 14, 2015 2:39 PM
To: 'Midrange Systems Technical Discussion'
<midrange-l@xxxxxxxxxxxx>
Subject: VTL and ProtecTIER deduplication
I'm referencing Redbook - IBM ProtecTIER Implementation and Best
Practices Guide
1) Anyone using ProtecTIER deduplication?
I'm curious how much savings I might see, especially large history
files that do not change.
As data is written to the ProtecTIER device, it is examined for
identical blocks of information that already were added to the repository.
This identical data is not stored again in the repository; it is referenced
as duplicate data and reduces the amount of disk space that is required.
This process is known as deduplication. The engine for ProtecTIER
deduplication is called HyperFactor.
2) Is there anyway of estimating how much storage is needed?
I currently have Perm Retention - 77 full LTO5 - 77 x 3.0 tb
(compressed) = 231 tb.
Create only the number of cartridges that your repository can handle,
maybe even fewer to control the repository allocation of different VTLs.
You can estimate the size of a repository by multiplying the real size of
the repository by the HyperFactor ratio. Then, divide it by the tape size
and determine the optimized number of tapes.
Important: Be careful not to overestimate the repository size. Wait
until the backup application sends some data to provide a better
view of the real deduplication ratio
3) Can existing LTO5 be migrated?
4) Will this change anything within BRMS?
5) Would I need an additional FC adapter on each LPAR to attach to
the I, multiple LPARs?
6) Can multiple processes be run simultaneously from multiple LPARs
as I do now with 4 LTO5 HH?
7) Would a VTL be faster than LTO5, LTO6, LTO7 (soon to be
announced)
18.2.1 Backup considerations with ProtecTIER Using VTL is not
necessarily faster than physical tape backup. IBM tape products have been
tested and work efficiently with IBM i. IBM i is able to achieve 90% - 100%
of tape drive speed in an environment with fewer tape drives. You often
require multiple streams in a VTL to achieve the same performance
throughput as physical tapes. In this scenario, Backup, Recovery, and Media
Service (BRMS) is useful in managing the tape media resources for parallel
saves.
In addition to performance throughput, you can use BRMS to share VTL
resources across multiple LPARs.
BRMS tracks what you saved, when you saved it, and where it is saved.
When you need to do a recovery, BRMS ensures that the correct information
is restored from the correct tapes in the correct sequence.
Thank You
_____
Paul Steinmetz
IBM i Systems Administrator
Pencor Services, Inc.
462 Delaware Ave
Palmerton Pa 18071
610-826-9117 work
610-826-9188 fax
610-349-0913 cell
610-377-6012 home
psteinmetz@xxxxxxxxxx<mailto:psteinmetz@xxxxxxxxxx>
http://www.pencor.com/
--
This is the Midrange Systems Technical Discussion (MIDRANGE-L)
mailing list To post a message email: MIDRANGE-L@xxxxxxxxxxxx To
subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx Before posting, please
take a moment to review the archives at
http://archive.midrange.com/midrange-l.
________________________________
Please consider the environment before printing this email.
The content of this e-mail (including any attached files) is
confidential and may be privileged and protected by law. It is intended
solely for the purpose of the person to whom it is addressed. If you are
not the intended recipient of this message, please notify the sender
immediately and delete the message (inclusive of any attached files). In
addition, if you are not the intended recipient of this message, any
disclosure, copying, distribution or taking any action in reliance of the
contents of this email is strictly prohibited.
--
mailing list To post a message email: MIDRANGE-L@xxxxxxxxxxxx To
subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx Before posting, please take
a moment to review the archives at
http://archive.midrange.com/midrange-l.
--
list To post a message email: MIDRANGE-L@xxxxxxxxxxxx To subscribe,
unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx Before posting, please take
a moment to review the archives at
http://archive.midrange.com/midrange-l.
--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing
list To post a message email: MIDRANGE-L@xxxxxxxxxxxx To subscribe,
unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx Before posting, please take
a moment to review the archives at
http://archive.midrange.com/midrange-l.
--
list To post a message email: MIDRANGE-L@xxxxxxxxxxxx To subscribe,
unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx Before posting, please take a
moment to review the archives at http://archive.midrange.com/midrange-l.
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing list
To post a message email: MIDRANGE-L@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/midrange-l.
As an Amazon Associate we earn from qualifying purchases.
This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].
Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.