Overview
Available for customers of Enterprise Storage (Dell Isilon) or the Center for Research Computing (CRC), the Archiving Service enables University departments and groups to easily move their seldom accessed data to more cost-effective cloud storage. Archive storage costs significantly less than primary storage (Dell Isilon and CRC) — allowing users to free up space on existing file shares while lowering overall storage costs.
Archived data can be retrieved. However, it is important to note that there are costs and waiting times associated with file retrievals. Therefore, data archiving is an ideal solution for rarely used files that still need to be saved, but not for files that need to be regularly accessed.
Several cloud storage options are available for archiving customers: Azure and AWS are supported. For AWS, we offer the options Standard (AWS S3 Glacier Instant Retrieval) and Deep (AWS S3 Glacier Deep Archive). Both services provide reliable and safe data archiving, however, Deep Archive offers greater cost savings for users who are willing to accept longer retrieval times and longer time requirements for how long their data must be kept in archive. In Azure, a Standard option is provided: Blob Storage Cold Tier. Discounted storage and retrieval costs are available with Azure. Additionally, with Azure Blob Storage Cold and AWS S3 Glacier Instant Retrieval, the Enterprise Data Transfer Service is available as an add-on enabling access to archived files.
Key Benefits
- Archive Files Automatically or Manually Select: Files that have not been accessed within a specified time (two years by default) get automatically archived in the cloud. Alternatively, users can select, and archive files as needed. These choices can be updated as needs change.
- Clean Up Data Clutter: Archiving rarely used data cleans up a file share and keeps important files that need to be regularly accessed front and center.
- Search and Retrieve Files: Search for archived files by reviewing the reports saved on the department's Enterprise Storage (Dell Isilon) or CRC file share. To retrieve archived files, simply contact the Pitt IT Storage Team with a list of identified files. For Standard Archiving, you can expect retrievals to begin within a day of the request. For Deep Archiving, retrievals begin next business day. Self-service retrieval of archived files is available with the Enterprise Data Transfer Service.
- Reduce Storage Costs: Storing data with Archiving Service and cost significantly less than primary Enterprise Storage (Dell Isilon) and the CRC.
| |
Primary Enterprise Storage |
Primary CRC Storage |
Azure Standard Archiving Service (Azure Blob Cold Tier) |
AWS Standard Archiving Service (S3 Glacier Instant Retrieval) |
AWS Deep Archiving Service (S3 Glacier Deep Archive) |
| Price per GB/month |
$.03 |
$.005 |
$.0036 |
$.004 |
$.00099 |
| Price per TB/month |
$30.72 |
$5.42 |
$3.69 |
$4.09 |
$1.01 |
Detail
Example
For a department that has 40 TB of data, 7 million files, located on Enterprise Storage (Isilon). The monthly cost to store this data currently is $1,229. The Archiving Service reports to the department that 25% of the data (10 TB) would be a candidate for archival as it has not been accessed in over 2 years.
The targeted 10 TB to be archived currently costs $307 per month to store on Isilon. Through Archiving Service, this same 10 TB of data would cost $34 if stored through the Azure Standard Archiving Service, $41 per month with the AWS Standard Archiving Service, or $11 per month with the AWS Deep Archiving Service.
The example below describes the costs for archiving and retrieving the full 10 TB. In most cases, only a portion of the data would need to be retrieved and can be done at lower cost based on size of data and number of files.
| Table reflects costs based on the 10 TB of data presented in the above example. |
| |
Cost per month |
One-Time Archive Move Fee* |
Cost per year (Year 1) |
Year 1 Cost Savings |
Cost per year (Year 2+) |
Year 2+ Cost Savings |
One-Time Retrieval Cost |
Time to Retrieve |
Minimum Storage Term** |
| Enterprise Storage (Isilon) |
$307 |
— |
$3,684 |
— |
$3,684 |
— |
— |
— |
— |
| Azure Standard Archiving Service (Azure Blob Cold Tier) |
$34 |
$39 |
$447 |
$3,237 |
$408 |
$3,276 |
$306 |
Same business day |
90 days |
| Standard Archiving Service (AWS Glacier IR) |
$41 |
$39 |
$531 |
$3,153 |
$492 |
$3,192 |
$1,247 |
Same business day |
90 days |
| Deep Archiving Service (AWS S3 Glacier Deep Archive) |
$10 |
$89 |
$209 |
$3,475 |
$120 |
$3,564 |
$993*** |
48-hour delay |
180 days |
*There is an initial, one-time fee associated with moving data to an archival storage tier.
**All archived data is subject to costs associated with minimum storage terms. Data recovered before the end of this set term will be charged for the term's full length.
***Expedited retrieval (12-hour delay) is available at an increased cost. In this example, the expedited retrieval would cost $1,858.
Minimum Billing Period Details
Objects that are archived to either Azure Blob Cold Tier or AWS S3 Glacier Instant Retrieval are charged for a minimum storage duration of 90 days. AWS S3 Glacier Deep Archive has a minimum storage duration of 180 days. Objects deleted prior to the minimum storage duration incur a pro-rated charge equal to the storage charge for the remaining days. Objects that are deleted, overwritten, or transitioned to a different storage class before the minimum storage duration will incur the normal storage usage charge plus a pro-rated storage charge for the remainder of the minimum storage duration. Objects stored longer than the minimum storage duration will not incur a minimum storage charge. For each object that is stored in S3 Glacier Flexible Retrieval or S3 Glacier Deep Archive, Amazon S3 adds 40 KB of chargeable overhead for metadata, with 8 KB charged at S3 Standard rates and 32 KB charged at S3 Glacier Flexible Retrieval or S3 Deep Archive rates.
| |
Primary Storage |
Azure Standard Archiving Service (Azure Blob Cold Tier) |
Standard Archiving Service (S3 Glacier Instant Retrieval) |
Deep Archiving Service (S3 Glacier Deep Archive) |
| Minimum Billing Period |
No minimum, delete data on demand
|
90 days
|
90 days
|
180 days
|
| Retrieval Start Time |
Data is immediately available.
|
Transfers begin the same business day.
Self-service access to download archived files anytime using the optional add-on of the Enterprise Data Transfer Service. Limitations apply.
|
Transfers begin same business day.
Self-service access to download archived files anytime using the optional add-on of the Enterprise Data Transfer Service. Limitations apply.
|
Transfers begin next business day.
Self-service access to archived files with the Enterprise Data Transfer Service is not available.
|
Monitor Archived Data with Detailed Reports
Access reports generated in the Tableau dashboard, as well as CSV spreadsheet reports, that helps to track and analyze your data to make decisions on how to best control storage costs.
Tableau Reports
This reporting shows file utilization across the Enterprise/CRC primary and archive storage. The department may monitor these reports in Tableau and can subscribe to have these reports emailed on a defined cadence.
| Volume Statistics |
Learn about your files, see how much space is used, how much remains and details about your files usage. |
| Volume Growth Projection |
See how your files change over time. How fast is your storage growing? Plan your budget for future needs. |
| Volume Churn |
See a more granular view of your file change with day-to-day activities for how fast your data is growing to plan budget for future needs. |
| User Size List |
See how much data your users are consuming. This gives you the opportunity to engage with those top users to consider deleting or archiving files that are not needed anymore, reducing your costs to use the storage services. |
| File Age |
This allows you to see how much data is becoming stale and could be a candidate for future archival. |
| Primary vs. Archive Storage |
See how much of your data is primary storage vs archived. |
CSV Spreadsheet Reports
CSV Spreadsheet reports are accessible locally on a customer’s file share. These reports list the files that have been archived and the time of their archiving.
| Directories older than 2 years |
This report will allow a department to see which files would be archived. Anything not accessed in 2 years will be included in the report. This is generated as part of the initial consultation with a customer. |
| Upload Job to Archive Target |
This report is generated every time an archive occurs and will list every file that was archived so you can easily see which files were archived. |
| Archive Service Estimates Report |
This report is generated as part of the onboarding process. It will provide an estimate of archiving costs and potential savings of utilizing this service. |
| All Archived Files |
Easily see a listing of all files that were archived. It will be in the folder “Data Management Service Reports” in the root of the share. |
Get Started with Archiving Service
Enterprise Storage or CRC customers can contact the Technology Help Desk to begin their onboarding process — outlined below.
- The Pitt IT Storage Team conducts an initial consultation with an interested customer to evaluate needs and requirements.
- Pitt IT analyzes the customer’s existing file collection and creates a report that includes a cost estimate based on the customer’s file count and sizes.
- The customer can use this report to determine if they will move forward with Archiving Service, determine if manual or automatic archiving is right for their needs, and help identify the files to be archived.
- After archiving, the customer will receive an email notifying them that a new archive report is available. This report will be stored in their file share and list every file that was moved to the cloud.
- For data on an automatic archiving plan (i.e., data older than 2 years), you will continue to receive reports when archiving occurs, and a report stored in your file share and listing every file that was moved to archive storage.
If a department is interested in Archiving Service, but is not a current CRC or Enterprise Storage customer, please contact the Technology Help Desk and a member of the Pitt IT Storage Team will contact to schedule a free consultation regarding their storage needs.