"blocked by CORS policy"

18 Dec, 2022

When I started building web applications that communicated with external APIs, I quickly encountered CORS errors. Some public APIs would work with no errors, like the catalogue of beers API that I used in my Punk API project. However when I attempted to fetch from a different API, in this case the electric vehicle charge point API I used in my charger-finder project, the GET request would succeed in postman but would throw an error when requested by my web application in a browser.

Here I try to answer:

Why do certain APIs work in postman, but not in the browser?
How can we use resources without access-control headers?

Why do certain APIs work in postman, but not in the browser?

There are two parts to a HTTP data exchange- a request and a response. Both of these contain headers. Headers are a type of metadata- data about data.

A server, like www.api.punkapi.com, has certain policies that they tell the client about using response headers. One of these properties is access-control-allow-origin, which describes what origins are allowed to access the resource.

To illustrate, let's make a request to https://api.punkapi.com/v2/beers/random. This endpoint fetches a random beer from the catalogue.

In the body of the response we get back our beer:

[
  {
    "id": 254,
    "name": "Small Batch: Kellerbier",
    "tagline": "Crisp Unfiltered Goodness.",
    //continues...

If we take a closer look at the headers of the response, along with others we see three headers that begin with access-control:

[
  {
    "key": "access-control-allow-credentials",
    "value": "true"
  },
  {
    "key": "access-control-allow-origin",
    "value": "*"
  },
  {
    "key": "access-control-expose-headers",
    "value": "x-ratelimit-limit,x-ratelimit-remaining,content-length,origin,content-type,accept"
  }
]

Now, let's consider access-control-allow-origin.

access-control-allow-origin is a list of origins that the server says are able to access the resources in addition to the origin itself.

For example, if a user is looking at www.mywebsite.com the browser is happy to allow requests be sent to www.mywebsite.com/getUserData. This is a same origin request and is generally very safe.

However, if the user is looking at www.evil.com and a script on the page silently makes a request to www.mywebsite.com/getUserData this is a little suspicious. This is a cross origin request and is one way an attacker can retrieve data from a vulnerable API endpoint, and store it elsewhere for malicious use.

By default the browser will not allow cross origin requests, unless the server specifically allows them using the access-control-allow-origin header.

The Punk API provides a wildcard symbol (star) therefore in an application like my Punk API front end project, the fetch requests are allowed without any issue.

Let's now make a request to the charge point API at the following address: https://chargepoints.dft.gov.uk/api/retrieve/registry/lat/51/lng/0/limit/3/dist/10/format/json

This says "find the 3 closest chargers to latitude 51 longitude 0 with a distance limit of 10 miles."

Carrying out this request in Postman and upon inspection of the headers, this server returns no access-control headers. Therefore browsers will not allow a webpage to send requests to this server, throwing the following error:

Access to fetch at 
'https://chargepoints.dft.gov.uk/api/retrieve/registry/lat/51/lng/0/limit/3/dist/10/format/json/lat/51/long/0/dist/2/format/json'
 from origin 'http://192.168.1.137:5500' has been blocked by CORS policy:
 No 'Access-Control-Allow-Origin' header is present on the requested resource.
 If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

We cannot change the headers that the server sends in the response, therefore we cannot use this resource in a web application!

How can we use resources without access-control headers?

If you do not control the server, you cannot send the right headers to allow cross origin requests.

I overcame this in my charger-finder-front-end project by creating a proxy server, that acts as a middleman between my web application and the public charge point API.

This took the form of a Spring Boot Application, which exposes an API endpoint. It is deployed using Google Cloud Run.

When a request comes in, Spring Application in turn requests data from the public charge point API. Since the Spring Application does not care about CORS policies in this circumstance, this is successful. Here is the function that I use to fetch from the charge point API, and map the response onto an object:

    @GetMapping("/chargers")
    public List<ResponseObject.ChargeDevice> getChargers(@RequestParam float lat, @RequestParam float lng) {
        String uri = "https://chargepoints.dft.gov.uk/api/retrieve/registry/lat/" + lat + "/long/" + lng + "/limit/3/dist/10/format/json";
        RestTemplate restTemplate = new RestTemplate();
        String result = restTemplate.getForObject(uri, String.class);
        ObjectMapper mapper = new ObjectMapper();
        ResponseObject chargers = null;
        try {
            chargers = mapper.readValue(result, ResponseObject.class);
        } catch (Exception e) {
            e.printStackTrace();
        }
        return
                chargers.getChargeDevices();
    }

The full GitHub repo for more context

Using a proxy server in this way brings two benefits:

We now control what headers are sent to the browser. We define our own access-control-allow-origins header, and therefore allow our browser based web applications to fetch.
The department for transport charge point API returns a lot of redundant information. The proxy server only passes on exactly what is needed, therefore reducing the amount of unnecessary data transfer to the client. Good for clients that have poor data signal or speed!

This post has been a few notes on what I have learned about CORS policies. It just scratches the surface of HTTP headers, and how we can use them to control who accesses our API endpoints.