Omeka continues to be a huge draw for a variety of students, faculty, and librarians using Reclaim Hosting. And the good folks at the Roy Rosenzweig Center for History and New Media have been champions of our service from the beginning, and that has made a huge difference for us. One of the issues that has come up regularly is storage for Omeka sites, which by design usually have large archives of documents, images, etc. We tend to keep our storage space for our Student and Faculty plans fairly low (2 GBs and 10 Gbs respectively) because we are trying to keep costs low, and the sales line of “unlimited” storage space for shared hosting is impractical for us. We recently introduced an Organization plan that has 100 GBs for just these instances because the need is there.
That said, if you have a lot of resources you might be better off with a service like Amazon’s S3—the backup redundancy is insane and you can’t beat the price. There are also lots of S3 compatible storage options offered by other vendors that may be even more price competitive, like Backblaze B2, but we will focus on AWS for the purposes of this how-to.
Setting up your S3
First and foremost you need an Amazon S3 account. S3 is cloud storage, and it is remarkably affordable if you have a large collection of files. That said, if you are Reclaiming from the corporate giants of the web, well, then…. Once you have abandoned your principles and signed up for an account—it’s easy, right?—you will need to do two things in AWS.
Next, you need to create an S3 bucket for your site where all the files will be uploaded. Heads up, when choosing a region outside of the U.S. (for example, Ireland is chosen in the example below) you need to define the endpoint URL in the config.ini file for Omeka. If your files are stored on Amazon servers within the US, this will not be a concern. More on that below.
After naming and choosing a region for your bucket, you will want to disable the 5 checkboxes blocking public access, and then check the box to acknowledge that objects in this bucket will be public:
You can keep all other settings while creating your bucket the same, and scroll to the bottom of the page and click Create bucket.
After creating your bucket, you will need to make sure all objects in the bucket are publicly accessible. Click on the name of the bucket you just created and go to the Permissions tab to get started. Next, scroll down to Bucket policy and click the Edit button.
In the Policy box, paste the following code, but be sure to replace BUCKET_NAME with the name of your AWS bucket you just created:
After you've pasted the code and put in your bucket name, click Save changes.
Note that if you get an error when saving your Policy, you may need to click the arrow next to API response, to see what went wrong. In this example below, the policy had a space as the first character, so double and triple check the policy if you are having trouble.
Creating user credentials
Now that your bucket is set up, you need to create a user for that Omeka can use to read and write files to the bucket. You do this in the Identity & Access Management area of the Amazon AWS dashboard, also called IAM. Use the search bar to go to the IAM section.
From the IAM area you click on Users in the left sidebar, then Add users. Give the user a name, and check the box next to Access key - Programmatic access then click Next.
Next, you will want to give the user S3 write permissions. You can click Attach existing policies directly, then search for the AmazonS3FullAccess policy. Check the box next to the policy, then click Next.
For the remaining steps, you can keep the default options and create the user account.
Once you do, you will have an Access Key ID and Secret Access Key similar to those shown below. Once you have the necessary keys, be sure to copy and paste these somewhere, you will need them for the Omeka config.ini file.
Once you do this, you are done in Amazon. You have got your bucket set up with the correct permissions, and you have the user created that will be used to authenticate into that bucket from Omeka. To finish the setup you need to go into your Omeka install and navigate to config.ini file (applications/config/config.ini)—File Manager may be the easiest way to get there in CPanel. Once there, you need to edit the file and uncomment the following section (which means removing the “;” in front of the code).
You’ll notice the AWS user Access Key and Secret key have been added, as has the bucket name. The other bit is the endpoint, which is only mandatory if you are storing files on a server outside the U.S. If so, you will need a URL like https://s3-eu-west-1.amazonaws.com as a defined endpoint so the Bucket will work. If you don't know your endpoint you can check out Amazon's article here on how to find it. You can also set an expiration time in minutes if you need it, if not, it can remain commented out.
Once you save the config.ini files with your access key, secret, bucket name, and endpoint (if needed) you should be all set. When you upload a file to Omeka now, it will be sent directly to this S3 bucket.
Additionally, if you already have images setup on Omeka, but would like to start using the S3 integration. You would need to copy all of your uploaded files (and the existing file structure) to that S3 Bucket. This is probably most easily done via FTP. Once you setup the config.ini file to point to S3, all existing file uploads on your site will break, so be sure to move everything ahead of time before you make S3 the default space for uploading files through Omeka.