Body
Overview
Pitt Digital's Archiving Service is available to departments and groups using Enterprise Storage (Dell PowerScale) or the Center for Research Computing (CRC). It allows you to move rarely used data to lower-cost cloud storage while keeping it safely preserved.
Archiving costs significantly less than primary storage and helps free up space on existing file shares. Archived files can be retrieved when needed, but retrieval will take time and incur additional costs. Archiving is best suited for data that must be kept but is no longer used or is accessed infrequently - not for files used on a regular basis.
Storage Options
The service supports cloud archiving in either AWS and Azure:
AWS
- Standard Archiving (S3 Glacier Instant Retrieval)
- Higher storage cost
- Fast easy access to files
- Retrieve your own files or request bulk retrieval by Pitt Digital
- Deep Archive (S3 Glacier Deep Archive)
- Lowest storage cost
- Longer wait to retrieve files
- Request retrieval by Pitt Digital
Azure
- Blob Storage Cold Tier
- Education and research discounts applied to all storage and retrieval cost
- Fast easy access to files
- Retrieve your own files or request bulk retrieval by Pitt Digital
Pitt Digital's Enterprise Data Transfer Service is included and enables browsing of archived files.
Key Benefits
- Automatic or Manual Archiving: Files that have not been accessed for a set period (two years by default) can be archived automatically. Files can also be selected and archived by placing them in a designated folder where all contents are archived regardless of age. These settings can be changed as needs evolve.
- Reduce File Share Clutter: Archiving removes rarely used data from file shares, keeping frequently used files easy to find and access.
- Find and Retrieve Archived Files: Archived files can be identified using reports saved on Enterprise Storage (Dell PowerScale) or CRC file shares. Pitt Digital's Enterprise Data Transfer Service enables self-service browsing of archived files. To retrieve files, simply create a Technology Help Request with a list of requested items.
- AWS S3 Instant Retrieval: File retrieval typically begins within one day
- AWS S3 Deep Archive: File retrieval begins the next business day
- Azure Cold Tier: File retrieval typically begins within one day
- Lower Storage Costs: Archiving storage costs significantly less than primary Enterprise Storage (Dell PowerScale) and CRC storage.
| |
Primary Enterprise Storage |
Primary CRC Storage |
Azure Standard Archiving Service (Azure Blob Cold Tier) |
AWS Standard Archiving Service (S3 Glacier Instant Retrieval) |
AWS Deep Archiving Service (S3 Glacier Deep Archive) |
| Price per GB/month |
$.0146 |
$.005 |
$.0036 |
$.004 |
$.00099 |
| Price per TB/month |
$14.58 |
$5.42 |
$3.69 |
$4.09 |
$1.01 |
Detail
Example
For a department that has 40 TB of data, 7 million files, located on Enterprise Storage (PowerScale). The monthly cost to store this data currently is $597. The Archiving Service reports to the department that 25% of the data (10 TB) would be a candidate for archival as it has not been accessed in over 2 years.
The targeted 10 TB to be archived currently costs $149 per month to store on PowerScale. Through Archiving Service, this same 10 TB of data would cost $34 if stored through the Azure Standard Archiving Service, $41 per month with the AWS Standard Archiving Service, or $11 per month with the AWS Deep Archiving Service.
The example below describes the costs for archiving and retrieving the full 10 TB. In most cases, only a portion of the data would need to be retrieved and can be done at lower cost based on size of data and number of files.
| Table reflects costs based on the 10 TB of data presented in the above example. |
| |
Cost per month |
One-Time Archive Move Fee* |
Cost per year (Year 1) |
Year 1 Cost Savings |
Cost per year (Year 2+) |
Year 2+ Cost Savings |
One-Time Retrieval Cost |
Time to Retrieve |
Minimum Storage Term** |
| Enterprise Storage (PowerScale) |
$149 |
— |
$1,792 |
— |
$1,792 |
— |
— |
— |
— |
| Azure Standard Archiving Service (Azure Blob Cold Tier) |
$34 |
$39 |
$447 |
$1,345 |
$408 |
$1,384 |
$306 |
Same business day |
90 days |
| Standard Archiving Service (AWS Glacier IR) |
$41 |
$39 |
$531 |
$1,261 |
$492 |
$1,300 |
$1,247 |
Same business day |
90 days |
| Deep Archiving Service (AWS S3 Glacier Deep Archive) |
$10 |
$89 |
$209 |
$1,583 |
$120 |
$1,672 |
$993*** |
48-hour delay |
180 days |
*There is an initial, one-time fee associated with moving data to an archival storage tier.
**All archived data is subject to costs associated with minimum storage terms. Data recovered before the end of this set term will be charged for the term's full length.
***Expedited retrieval (12-hour delay) is available at an increased cost. In this example, the expedited retrieval would cost $1,858.
Minimum Billing Period Details
Objects that are archived to either Azure Blob Cold Tier or AWS S3 Glacier Instant Retrieval are charged for a minimum storage duration of 90 days. AWS S3 Glacier Deep Archive has a minimum storage duration of 180 days. Objects deleted prior to the minimum storage duration incur a pro-rated charge equal to the storage charge for the remaining days. Objects that are deleted, overwritten, or transitioned to a different storage class before the minimum storage duration will incur the normal storage usage charge plus a pro-rated storage charge for the remainder of the minimum storage duration. Objects stored longer than the minimum storage duration will not incur a minimum storage charge. For each object that is stored in S3 Glacier Flexible Retrieval or S3 Glacier Deep Archive, Amazon S3 adds 40 KB of chargeable overhead for metadata, with 8 KB charged at S3 Standard rates and 32 KB charged at S3 Glacier Flexible Retrieval or S3 Deep Archive rates.
| |
Primary Storage |
Azure Standard Archiving Service (Azure Blob Cold Tier) |
Standard Archiving Service (S3 Glacier Instant Retrieval) |
Deep Archiving Service (S3 Glacier Deep Archive) |
| Minimum Billing Period |
No minimum, delete data on demand
|
90 days
|
90 days
|
180 days
|
| Retrieval Start Time |
Data is immediately available.
|
Transfers begin the same business day.
Self-service access to download archived files anytime using the optional add-on of the Enterprise Data Transfer Service. Limitations apply.
|
Transfers begin same business day.
Self-service access to download archived files anytime using the optional add-on of the Enterprise Data Transfer Service. Limitations apply.
|
Transfers begin next business day.
Self-service access to archived files with the Enterprise Data Transfer Service is not available.
|
Monitor Archived Data with Detailed Reports
Access reports generated in the Tableau dashboard, as well as CSV spreadsheet reports, that helps to track and analyze your data to make decisions on how to best control storage costs.
Tableau Reports
This reporting shows file utilization across the Enterprise/CRC primary and archive storage. The department may monitor these reports in Tableau and can subscribe to have these reports emailed on a defined cadence.
| Volume Statistics |
Learn about your files, see how much space is used, how much remains and details about your files usage. |
| Volume Growth Projection |
See how your files change over time. How fast is your storage growing? Plan your budget for future needs. |
| Volume Churn |
See a more granular view of your file change with day-to-day activities for how fast your data is growing to plan budget for future needs. |
| User Size List |
See how much data your users are consuming. This gives you the opportunity to engage with those top users to consider deleting or archiving files that are not needed anymore, reducing your costs to use the storage services. |
| File Age |
This allows you to see how much data is becoming stale and could be a candidate for future archival. |
| Primary vs. Archive Storage |
See how much of your data is primary storage vs archived. |
CSV Spreadsheet Reports
CSV Spreadsheet reports are accessible locally on a customer’s file share. These reports list the files that have been archived and the time of their archiving.
| Directories older than 2 years |
This report will allow a department to see which files would be archived. Anything not accessed in 2 years will be included in the report. This is generated as part of the initial consultation with a customer. |
| Upload Job to Archive Target |
This report is generated every time an archive occurs and will list every file that was archived so you can easily see which files were archived. |
| Archive Service Estimates Report |
This report is generated as part of the onboarding process. It will provide an estimate of archiving costs and potential savings of utilizing this service. |
| All Archived Files |
Easily see a listing of all files that were archived. It will be in the folder “Data Management Service Reports” in the root of the share. |
Get Started with Archiving Service
Enterprise Storage or CRC customers can contact the Technology Help Desk to begin their onboarding process - outlined below.
- The Pitt Digital Storage Team begins with a consultation to understand archiving needs and requirements.
- Pitt Digital then reviews existing files and provide a report with a cost estimate based on file size and volume.
- This report helps you determine whether to proceed with the Archiving Service, choose between automatic or manual archiving, and identify which files should be archived.
- Once archiving is complete, an email notification is sent and a detailed archive report is saved to the file share. This report lists all files moved to cloud storage.
- For automatic archiving (such as files older than two years), archive reports will continue to be generated and saved to the file share each time archiving occurs.
If a department is interested in Archiving Service, but is not a current CRC or Enterprise Storage customer, please contact the Technology Help Desk and a member of the Pitt Digital Storage Team will contact to schedule a free consultation regarding their storage needs.