I recently helped my customer successfully optimise their storage using IBM’s storage solutions. During the process, we had over 40TB of data lifted off the baremetal servers, and onto ICOS (IBM Cloud Object Storage). The result was amazing! High availability, ease of access, resilience and of course reduction in costs. I have realised though that often organisations are too busy with operations that taking additional time to reconsider the current architecture may be difficult.
The demonstration I did to help show the value of ICOS has been very helpful to the customer, so I decided to take a moment to document it here, I hope this helps for you too!

Quick definition on object storage:

ICOS is IBM’s Object Storage solution, and it is especially useful for storing unstructured data for archiving purposes. As this is a technical blog, I will keep explanation to minimum and get started with the demonstration.
About Object storage

Did you know? If you have an IBM Cloud account, you can provisiong Lite instances of ICOS for free! In this demo you can try to get your own instance up and running, or just get a picture of how it looks in the portal when you use ICOS.

DEMO for ICOS

Firstly, go to the catalogue to find ICOS:

Pick the Lite instance during order. Once it is provisioned, it will take you to the introduction page:

If you navigate to the ‘buckets’ page, you will see the following page. You can do all your administration here. In the beginning you will have 0 buckets:

Lets go and create a bucket. Click on the arrow for ‘Customize your bucket’ for full options:

The bucket name will have to be unique across the whole ICOS system. It’s a good practice to use multiple words separated by -.
Pick the resiliency depending on storage requirements. Single Site is best if you have certain data that cannot leave a particular geography, but has no resilience.
Regional has replication across Multi Zone Regions in the same location. Cross Region has best resiliency, with replication across geographies (it’s good to understand your budget too!)

There are several storage classes. Standard is best for high access workloads, and has no charges for data retrieval. Vault is in between and Cold vault is best for long term archiving, which is low cost but has charges for retrieval. Generally the long term archiving tiers would take longer to retrieve the data. Smart tier is a great tier that will automatically allocate data into the correct tier depending on frequency of access, and help reduce cost. One note is that a bucket cannot be changed to a different storage class, but you can create other buckets and move across data using command tools like rclone:

There are a few more options for the bucket, and once you have chosen the right configuration, your bucket is created! An additional setup I recommend is the service credentials. By adding service credentials you can access content in the bucket externally and securely.

When creating new credentials, click Advanced options, and check Include HMAC Credential to be ON.

Check the access key id and secret access key:

You can use 3rd party tools like Cyberduck to securely connect to the ICOS bucket by entering the access key id and secret access key as follows. ICOS uses the same interface as Amazon S3, so you can select that when prompted the type of storage. For server, enter the endpoints as found here.

I was able to connect to the bucket from Cyberduck. I can perform basic operations like adding file, retrieving etc.

Other useful features include the aspera high speed upload. You can do this if you are uploading large files from the portal. From the bucket after you click upload, select transfer type to be Aspera high speed upload, it will be blazingly fast! You will need to follow instructions and download the aspera client to use it.

Another useful feature is rclone, which is a tool that you can use on the commandline to move data from a bucket to another bucket. Generally you cannot move objects between buckets, so features like this help when you need to move data around. This feature supports any Object storage not just IBM, so for example if you have object storage on other cloud vendors it is possible to move it to ICOS too:

That’s it for the demonstration! There are more things to keep in mind including the following:

  • User permissions
  • Changing to the paid plan if you wish to convert to real workloads
  • Using API and commandline tools

I hope this served as a good intro to ICOS! :)

Reference and good reads/watch:

Page to create object storage instance on IBM Cloud

IBM Cloud Object Storage webinar

Access Management

Getting started with the IBM Cloud CLI

Storing large objects with CLI

Using the AWS CLI:

IBM Cloud Object Storage CLI

IBM Cloud CLI Quick Reference (includes the slack communication channel at the bottom for questions)

Transferring files with Cyberduck

Using rclone