Blog

Hosting a static website in S3 with a custom domain in Cloudflare providing free unlimited bandwidth

Published:

The Goal

This article will show you how to:

  1. Create an S3 bucket to host your static site
  2. Create an S3 bucket to re-direct traffic from https://mysite.com to https://www.mysite.com
  3. Add your domain to Cloudflare and setup DNS records and nameservers

...all for free and with unlimited bandwidth. I will document this process as I set it up for a real website.


Foreword

This section details why we're using this stack. If you just want to see the how, click here to skip ahead.

Why Cloudflare

Cloudflare, unlike AWS's similarly named Cloudfront, takes control of your domain and caches any content served from the source (in this case, an S3 bucket). Importantly, Cloudflare provides free, unlimited bandwidth on all of that cached content that it serves.

This means you only pay for the initial traffic out of S3. After the first load everything is cached (by default for 4 hours) and all those cached files are served in a globally distributed CDN for free. By comparison, AWS's Cloudfront charges around $0.08 per GB of data.

Example: your 1MB website at www.mysite.com has 1,000,000 visitors. With AWS's Cloudfront, you pay $0.08 per GB regardless of whether the file is cached or not. For one million hits you use $80 of bandwidth. With Cloudflare, you pay for the first hit, and the subsequent 999,999 are free. This is truly unlimited bandwidth even for large sites.

*This hypothetical assumes all your traffic comes from the same region (and within the cache TTL). Cloudflare have roughly 200 data centers scattered across the globe, so each time someone from a new region hits your site that datacenter will be populated with your cached content. Later in the article I will detail increasing the cache TTL so you have even less outbound traffic from your source.

Why S3

Because it's easy, cheap and convenient, but these principles will work on any blob storage provider so long they allow you to enable static hosting (e.g. Azure).


1. Create S3 buckets to host your site

We want all our traffic to end up at https://www.mydomain.com, even if the user types mydomain.com, www.mydomain.com, http://mydomain.com or any other variation.

To do this we will create two S3 buckets. Our main bucket will host our static website; the other will simply re-direct to that main bucket. We'll do these rest with Cloudflare's DNS records later.

1.1 Create the buckets

Go to AWS, navigate to the S3 admin panel and click create new bucket.

The name of the first bucket will be the full domain name, beginning with www. but excluding any https:// prefix.

Select your region and make sure to uncheck the "Block all public access" option:

https://i.imgur.com/xgA9pfT.png

AWS will present a warning dialog when you make a bucket with public access - accept it. You can also choose whether you want versioning on or not. It's disabled by default, but I am going to enable it so that each file has a previous version available in case of accidental overwrites.

https://i.imgur.com/Fu7C0IC.png

The rest of the options can be left to their default value. Click "Create Bucket" to continue.

1.2 Enable static hosting

With the main bucket created, we now need to enable static web hosting. This setting means AWS will serve the contents of the bucket as webpages to be rendered rather than files to be downloaded.

To enable it, open the bucket from your list and go to the "Properties" tab:

https://i.imgur.com/pv0B0oo.png

Scroll to the very bottom and you will see a "Static website hosting" box. Click the "Edit" button, then select "Enable" on the dialog that appears.

https://i.imgur.com/J9dKBvL.png

In the above image you can see that I have entered the default index.html as the index document, but I have also entered index.html as the error document. This makes life easier if we want to host a single page app like React; due to React's routing system, sharing direct URLs can be tricky. Having the error document as index.html means that a link to www.mysite.com/settings/edit will get routed to the index.html (because no /settings/edit directory exists in our SPA) - React's router can then handle the URL from there.

Leave everything else as is and click save.

1.3 Restrict access to Cloudflare

We don't want people to access our bucket directly - that would bypass Cloudflare's caching and is generally something we want to restrict.

Go to the permissions tab:

https://i.imgur.com/gIsHMoT.png

...and scroll down to the "Bucket policy" pane. Click "Edit" and enter the following policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PublicReadGetObject",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::www.YOURDOMAINNAME.com/*",
            "Condition": {
                "IpAddress": {
                    "aws:SourceIp": [
                        "2400:cb00::/32",
                        "2405:8100::/32",
                        "2405:b500::/32",
                        "2606:4700::/32",
                        "2803:f800::/32",
                        "2c0f:f248::/32",
                        "2a06:98c0::/29",
                        "103.21.244.0/22",
                        "103.22.200.0/22",
                        "103.31.4.0/22",
                        "104.16.0.0/12",
                        "108.162.192.0/18",
                        "131.0.72.0/22",
                        "141.101.64.0/18",
                        "162.158.0.0/15",
                        "172.64.0.0/13",
                        "173.245.48.0/20",
                        "188.114.96.0/20",
                        "190.93.240.0/20",
                        "197.234.240.0/22",
                        "198.41.128.0/17"
                    ]
                }
            }
        }
    ]
}

You must change the bucket name on line #9 to the name of the bucket you just created. This policy lists all the IP addresses that Cloudflare's reverse proxy uses and enacts a policy which states that only traffic from those IP addresses can retrieve content from your bucket.

Without Cloudflare, our traffic would flow directly from the user to the S3 bucket, meaning the incoming IP would be from the user. With Cloudflare, however, the traffic flows from the user to Cloudflare and then from Cloudflare to our bucket, with the content being cached in Cloudflare's network on the return journey. Any future requests will be returned directly from Cloudflare's cache without hitting our bucket until that cache expires (4 hours by default, although this can be edited to as long as a year in Cloudflare's settings).

1.4 Upload your content

At this stage you can upload your website's files to the bucket. If you just want a proof of concept, upload the following index.html file:

index.html:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Document</title>
</head>
<body>
    <h1>Hello, world!</h1>
</body>
</html>

Go back to the "Objects" tab in the S3 Console:

https://i.imgur.com/oOfpV1G.png

...then click the "Upload" button and select the index.html file you just created. You can also drag and drop files into the browser directly.


2. Create the redirect bucket

We want all our traffic to end up with the domain https://www.mysite.com. Whilst we can enforce HTTPS in Cloudflare, what if a user goes to https://mysite.com? To handle that redirect, we are going to create another bucket called mysite.com that will redirect to the www.mysite.com bucket we created in the last step.

2.1 Create the bucket

Create the bucket exactly as before but this time do not enter the www. prefix:

https://i.imgur.com/gCmDz5q.png

2.2 Redirect to the www. bucket

Again go to the "Properties" tab and scroll to the "Static website hosting" panel at the bottom. Click edit and enable static hosting.

This time, however, we are going to redirect to the bucket that we created in step #1. Select "Redirect requests for an object" and in the host name textbox enter the name of the bucket we first created:

https://i.imgur.com/hJFRXdX.png

For good measure, I am also going to select the HTTPS protocol. I am not sure if this enforces a redirect for HTTP to HTTPS or if it only performs the redirect on HTTPS traffic, but either way it's good to enable it congruent to the "HTTPS everywhere" goal.

And that's it - click save and our work in AWS is complete.


3. Add your custom domain to Cloudflare

3.1 Create site in Cloudflare

Go to Cloudflare and either login or create a new account. There is no need to pay for anything; we will only be using the free plan. And yes, unlimited bandwidth is included in the free plan (amongst many other features) - this is why Cloudflare is so great.

https://i.imgur.com/0APSW0o.png

In the dialog box that appears, enter your website without any https://www. prefix.

You will be presented with a plan selection window that tries to guide you towards a paid plan. Select the free option at the bottom:

https://i.imgur.com/KikslW7.png

...and click next.

3.1 Add DNS records

Cloudflare will then perform a scan to determine your domain's existing DNS records. The records we are concerned about are the CName and A records; if any existing records of this type are found, delete them.

We will start by creating a CName record for the main bucket (the one beginning with www.). Click the "Add record" button and select "CName" as the type. In the target box, we want to enter the endpoint URL for the main bucket. You can find this by going to the "Properties" tab in the S3 Console:

https://i.imgur.com/RWBi3GG.png

...and scrolling to the very bottom to find the "Static web hosting" panel:

https://i.imgur.com/OBk9PoE.png

However, if you can't be bothered doing that you can simply generate the URL from the following naming convention;

[bucketname].s3-website-[region].amazonaws.com

When entering the target into Cloudflare, make sure you omit the http://. It should look like this:

https://i.imgur.com/7KmZ0RN.png

Leave the other default options and click save.

Next, we will create another record for the redirect bucket. The process is the same, except this time the CName record will be yoursite.com without any https://www.:

https://i.imgur.com/V4UsJ0Z.png

...and the target will be the redirect bucket, so the same target as before except without the www..

We should now have two CName records, each pointing to one of our buckets. Double check the naming of each CName record and click next.

3.2 Change your nameservers to Cloudflare

The next step is to change your domain's nameservers so that they point to Cloudflare's.

Go to your domain registrar and log in. I am using Namecheap but the process will be much the same at any registrar. With Namecheap, you have a list of domains and a "Manage" button:

https://i.imgur.com/hKDKqEM.png

You will then see the existing nameservers. Delete them and replace them with the nameservers that Cloudflare presented to you.

https://i.imgur.com/WhHHjL9.png

Click save.

3.3 Check nameservers

Go back to Cloudflare and click the "Done, check nameservers" button:

https://i.imgur.com/hJ9JjeH.png

That will kick off in the background. While that check is performing, Cloudflare will give you some options:

You will then be returned to the Cloudflare admin panel for your newly created site.

3.4 Re-check nameservers

If the nameserver check did not succeed, click "Check nameservers" to begin the check again. It can take up to 24 hours for your domain registrar to process the change but it most cases it goes through much quicker.

https://i.imgur.com/XCykbEP.png

Try refreshing the page after a few minutes; hopefully you see this message:

https://i.imgur.com/HGbOryO.png

3.5 Change SSL mode

S3 does not support SSL certificates, so we will have to change the SSL settings in Cloudflare.

On the Cloudflare panel, click SSL/TLS:

https://i.imgur.com/DSDxXdj.png

Then change the mode to flexible:

https://i.imgur.com/X0G6EIX.png

This means the content coming from S3 to the Cloudflare cache will not be encrypted (it will be encrypted between Cloudflare and your end users, however). Regardless, this is only static content and there should be no security concern here. Any backend interaction done on your website will happen with some other service (e.g. Lambda / API Gateway), which is unaffected by Cloudflare.

3.6 Increase cache TTL

The default cache time-to-live is 4 hours. That means that once your content has been cached it will sit there serving your visitors for 4 hours until re-fetching the content from your source once that period has elapsed. This is OK as S3 costs are cheap, but for a static site like the one I am setting up it's unnecessarily short given that the content changes so infrequently. There's no point filling up all 200 of those datacenters every 4 hours if the content hasn't changed.

To increase this period, click "Caching":

https://i.imgur.com/dndWaLP.png

...then click "Configuration":

https://i.imgur.com/1BBa3XO.png

...and scroll down to "Browser cache TTL":

https://i.imgur.com/dmeaE1W.png

I have set mine to 16 days, but it can be as high as a year.

Be aware that updating your content in S3 will not automatically clear the Cloudflare cache. This will have to be done manually by purging the caches:

https://i.imgur.com/bgOHzoX.png

3.7 Troubleshoot

This should work for you. It worked for me as I documented this process and this is now the third time I have done it. However, if you do have issues here are a few tricks you can try.

Make files public during upload

This shouldn't matter, but you can try re-uploading your website and making the files public during the upload dialog:

https://i.imgur.com/MfOw558.png

You can also try temporarily removing the bucket policy we created earlier to restrict S3 access to the Cloudflare IP addresses.

Thirdly, make sure you have changed the SSL settings in Cloudflare as leaving the default option will not work.


Finish

You should now have a website with a custom domain serving content from S3. Try going to different variations of your domain - www.mysite.com, http://mysite.com etc. - and they should all be redirected to https://www.mysite.com.

You may experience choppy performance for a few hours as DNS caches are updated. If you are transferring from an old webhost, it's normal to still receive the version of the website hosted on the old host for a few hours.