Synology’s Data Tiering Previewed in Channel Event
Choosing the right storage medium for your data is- challenging: You are looking to balance the speed of expensive flash storage with the affordability of slower HDDs. The key challange is not to decide which one to choose ( you can offcourse choose both), but rather how to ensure your data is stored on the most suitable medium.
Ideally, your HOT, CURRENT data (frequently accessed) should reside on fast flash storage, while COLD, ARCHIVAL data (infrequently accessed) can be housed on more cost-effective HDDs. That's called tiering of data.
Sounds simple - keep your hot data on SSD volume, and cold data on HDD volume, But once you do it you realize that manually maintaining this structure is cumbersome. Thats when you start looking at automating it with either Caching or Auto-tiering
Caching vs. Auto-Tiering: The Solution Lies in Auto-tiering
Caching addresses some performance bottlenecks but doesn’t solve the underlying challenge of optimal data placement. If your volume of HOT data exceeds the size of your cache the purpose is defeated (and you can't have as much cache as you want becuase the max cache you can have is tied to max RAM you have) . With lots of data, you need to rely on your primary flash and HDD volumes, and a way to ensure that the right data resides on teh right volume.
This is where auto-tiering comes into play. Auto-tiering continuously monitors data usage patterns and dynamically relocates data based on access frequency:
- Files that aren’t accessed frequently are moved to HDD.
- Files that are accessed repeatedly on HDD are promoted to flash.
Data migration between tiers takes time, so it’s typically scheduled. The determination of which files to move is driven by algorithms such as:Least Frequently Used (LFU), Least Recently Used (LRU) or a mix of both for better precision.
End result - dynamic allocation of data across multiple storage tiers based on usage patterns. It ensures that the right data resides on the right medium (most of the time, as usage pattern change you have to wait for next schedule to move data from one tier to another)
Synology’s Take on Data Tiering
Synology recently unveiled details about their upcoming data tiering feature. While QNAP has long offered a similar solution with Qtier, Synology is just entering this space. From the slides shown, here are some observations:
- Customizable Policies: Synology’s approach seems to include the ability to define policies, allowing users to set their own logic for data profiling and movement. We don't know what the policies would be but we can guess that they would probably let you pick the algo and maybe some files that you would want to remain on a tier irrespective of their usage.
- No Cloud Archival Tier: Unlike some systems that include a cloud-based archival tier, Synology appears to focus on multiple local volumes for tiering, at least that’s what was shown on slides.
- Single-Unit Tiering Unclear: It’s not evident from the slides if Synology will support tiering within a single unit or if it requires multiple units. In the slides they have shown tiers across devices
Synology’s entry into the auto-tiering is exciting , given their reputation for delivering thoughtful, user-centric solutions I am very keen to try their solution. The most interesting part being the customizable policies. Hopefully this will be introuced around Q1 25.