Written by Frederic Bourget
In a previous blog post, I discussed Moore’s law and the misperceptions that are often extrapolated from it. One of these perceptions was that the reduction of storage costs follows the same type of curve as the increase in transistor density. Over the last 30 years, it certainly felt that way. Hard drive capacity for consumer products kept increasing every year and prices stayed about the same. But for organizations, the reality is otherwise. For example, when as a company you are running an archive service, disk costs, and density are major contributors to operating costs. The idea that storage costs go down faster than documents accumulate is a myth. Keeping everything forever is a bad practice from compliance and from a risk perspective, but is also very expensive.
Cost Decrease Is Slowing Down
While the following data reflects the cost of hard drives from 2017, it is still prevalent in today’s current cost of hard drives. The team at BackBlaze, my favorite backup service, which manages over 600 TB of data, has compiled their costs of hard drives. They claim to offer the cheapest cloud storage in today’s industry with their B2 hardware. But they also observed that the rate at which the cost of hard drives is decreasing dramatically slowed in recent years and they start to see the limit to decreasing prices.
As you can see from the curve, the improvements in storage costs are tapering off. This means that savings from storage density will get smaller and smaller as we go forward. Meanwhile, our appetite for storage is following the opposite trend. Gartner predicts enterprise storage growth to be 50% year-over-year, or nearly 800% over 5 years. In other words, organizations’ exponential accumulation of data will no longer be compensated by an exponential decrease in storage prices, so the cost of keeping everything will begin to grow rapidly unless there is a breakthrough in storage technology.
The Limitations of Data Storage Technology
There are a number of really cool new techs in the work, but none are ready to hit the market. SSD really is the only one of these innovations available today. Based on flash memory, it follows Moore’s law. We could imagine that it would at a certain point compensate for the limitations of current magnetic disks. The curve below shows V-NAND memory that is very promising in terms of density increase. However, for flash memory to be adopted, it has to be cheaper than magnetic storage. Currently, the cost of magnetic drives is in the area of $0.045 per Gig where Flash storage is at $0.25. Considering that magnetic storage’s prices are still decreasing, it will take 5-10 years for these two types of memory to get into the same price range. But when exactly this will happen is hard to predict because as I explained in a previous post, the cost of manufacturing new tech is getting higher with each node increase. So we can assume that for the next little while, there will not be major disruptions.
Could Cloud Storage Be The Solution?
Storage in relation to data growth is not cheap… and won’t be for a while. So where does the industry stand in terms of cloud providers? Well a lot of vendors are pitching unlimited storage on their platform. They claim storage is cheap and it does not matter how much is dumped in their cloud. So why manage data on your own storage when you can just keep cluttering the cloud forever?
Because there’s a hidden agenda.
Most vendors know that once the data is in their cloud, it’s really hard to get it out. The more data is in there; the more likely organizations won’t want to move it. Some even go as far as limiting the amount of data that can be taken out every month or charging exorbitant egress and services fees. To keep their business, they make sure it will never make sense for anyone to leave. That’s what’s called a vendor lock-in. This is one the reasons I like our technology. IPRO has taken a different approach. Your data is yours, and you can take it out when you want. We want our customers to stay because they see the value we offer, not because they have to.
Busting The Cheap Storage Myth
So are storage costs going down faster than data accumulate? The answer is not what people want to hear. Cheap storage is a myth. Managing data is still, and will be for a while, a better strategy for organizations than to keep everything forever.
