IBM Cloud Storage
Intro
This is the fourth installment of the IBM Cloud series referring in more detail to the different storage services that we have had experience with on IBM Cloud. Once again this will refer mainly to the classic instance type of implementation as it is more close to a public cloud solution we see in other cloud suppliers.
When referring to storage, in IBM Cloud you will have to deal with a number of storage implementations like local disks, block storage, object storage and file storage.
Local drives
By default, most IBM Cloud instances on a classic implementation, whether they are public or private, have the so-called local disks. The disks in themselves are not local as they explicitly are stated as SAN attached. The term local mainly refers to the fact that these are the volumes on which the template will be provisioned and the subsequent OS installation will happen.
One important thing when planning the size of the instance and including there local disks, please be aware that the size of the disk of the instance, from which you created your template will be crucial when deploying the new instance afterwards. Unfortunately, there is no real choice to deploy the instance on a larger disk, when the template has been created from an instance with a smaller one.
As we mentioend before, local drives are not encrypted. This is maybe one of the biggest hurdles that you will find when attempting to deploy end-to-end encryption within IBM Cloud. The only solution that we found on the issue is creating a custom encrypted template, to use later for deployment of future instances.
Other than that, local SAN-attached drives are stable, well-performing and will most probably fit your needs if the size is not a limitation.
Block storage
If size is an issue, of course, you can always use the block storage service that IBM provides for its customers in the cloud. Block storage volumes are iSCSI-based chunks, which attach via the private network interfaces of the instance and have a few specifics.
Block storage is flash-backed and fully encrypted by default.
You should always bear in mind that you have no choice whether to add one, two or more network interfaces to your instance. The instance comes with a private (always) interface and with a public (optional) interface, which provides access to the Internet. There is no possibility of a 10Gbps interface or more interfaces. The option is 100Mbps or 1Gbps.
We would recommend always using 1Gbps interfaces as they are not rate-limited and will allow you to achieve maximum speed towards your block storage volumes.
By default the block storage volumes’ performance is determined by its size. Meaning that if you get a 100GB volume from a 10IOPS/GB tier, you will receive a piece capable of achieving 300 IOPS. Bearing in mind that the maximum size of block storage volume is 12TB, this should provide the ability to achieve a 120 000 IOPS, however, please note in most data centres of IBM Cloud you will find that the limitation there is 6000 IOPS.
There are two main types of block storage – endurance and performance. We suggest using endurance as performance gives access to custom performance levels, which in most cases are unnecessary.
When planning your block storage performance, two main things come to mind – data centre and network limitations of the instance itself.There are a few data centres, perhaps from the older generation of IBM Cloud, where block storage performs poorly in comparison to newer data centres. Be sure to test beforehand as it could cause a number of performance issues if performance is key for your application.
Network of hosts is of critical importance as you may expect from an iSCSI-attached volume. There are a few things you could do.
Besides the fact that the private network interface looks like 1Gbps when you ordered it, IBM Cloud will provide you up to 5Gbps speed on this interface if you manage to optimise it – applying jumbo frames on the OS level, adjusting queue depths etc. Be sure to measure the block size of your application, which may be the limiting factor from a throughput point of view.
File storage
If you need a NFS share for your instance, file storage is the way to go in order to avoid serving NFS from one of your instances.
File storage is a lot like Block storage in matters of capabilities and also it’s limitations. Once again you have it encrypted and in the same performance tiers, backed by flash.
One thing that we found a bit uncomfortable is the inability to mount file storage between the data centres. In a number of cases you need a small share between your servers in different locations, however, this cannot be done directly via File Storage. Of course you can mount to a server in one location and serve to others, however, this mostly defeats the purpose of avoiding a single bottleneck from availability and performance point of view.
Snapshots and replications
Snapshots of block storage work like a charm. They are very convenient if you need a tool to quickly replicate one of your block storage volumes and attach to another instance. They are instantaneous and can be done under a schedule. Use them often if you have big changes coming to your system, the cost is insignificant to the advantage that they give.
Replication to other data centres is based, as you may expect, fully on the snapshot technology. Snapshots schedule needs to be created in order for snapshots to be shipped to the other location and a functional copy to be available there. With the replicated snapshot, you can do a number of convenient things, like duplicating to a new volume and attaching to an instance there. Also a full revert of the replication is possible in the event of a disaster recovery type of procedure.
If there is one issue with the replication itself, it is the inability to choose scheduled periods smaller than 1 hour. On some applications, usually DBs, 1 hour RPO is not an acceptable limitation, so other tools should be used for cross-locations availability.
However overall snapshots and replications on block and file storage are convenient, quick and really easy to use.
Object Storage
Object storage is IBM Cloud global object storage service. Very useful when you are OK to store information as an object, need global reliability and do not have strict requirements on throughput and IOPS. Object storage can be cross region – meaning the information is copied onto three different regions, regional – in a number of data centers, or single site – which is on a single site, as expected.
As most object storage services go, it has hot, cool, cold tiers for performance as well as an API-based pricing. The Aspera tool is used to accelerate performance of IBM Cloud Object storage. Service is really convenient for storing large amounts of data – critical or not, even across regions.
Its API interface is really powerful and can be really helpful when attaching directly to a development pipeline.
Finally 🙂
This was our view on some of the main IBM Cloud storage services, with the remark of course that we cannot comment on all of them, but the ones that we use most frequently.
Do not hesitate to contact us, review our IBM Cloud services or just call us, if you need more help on your public cloud deployment projects.
Note: We are not aligned to any particular public cloud provider and are an independent cloud services provider. This blog is solely to share our experience and is based on real implementations.
Bye!
