Architecture and Design, Guest Blog /

The three secrets to protecting your API

Secure by Design banner

Can you imagine that only 5 years ago the first iPad was launched? It’s mind boggling thinking that my children are using technology that used to be part of science fiction literature. They haven’t known anything else and to them there is no novelty. On their iPads and smart phones my children have the world’s information at just a few gestures away. This mobile revolution not only changes how we interact with information but it also has an impact on the application technologies we use to service the variety of mobile devices.

Looking back at the web applications of ten years ago those applications have a tight coupling between frontend and backend. The heavy lifting is done in the backend where the data, business logic and presentation logic resides. The browser in this setup is merely a viewer of the screens that are generated by the backend. This is no longer the case with the modern mobile client where a lot of the heavy lifting has shifted to the front. The perennial cycles of IT have brought us back to the era of the fat client (for those of you who remember LAN based Windows applications), albeit in a somewhat different fashion than the fat clients of the past. For one thing the new fat clients are solidly embedded in web technology and are based on open standards. This new breed of mobile technology is known as Single Page Applications (SPA) and they are centered around using web technologies like javascript, html5 and cascading style sheets. This set of technologies enables rapid development, internet scale deployment and flexible integration. Where the classic web application requires an always on Internet connection these new applications stand on their own and can run independently from the backend. The new breed of mobile applications can work in disconnected situations by storing information on the device and exchanging information when an internet connection is available.

Separating the frontend and the backend has an enormous impact on the backend systems. No longer are these systems required to generate html, which in all honesty was never a happy marriage as backend engineers generally are not very focused on aspects of interaction design and user experience. In the new mobile application the backend system exposes its data and services via an Application Programming Interface (API). API’s have been around for decades in software development, the concept has been in use since the 1970s. These days when the IT crowd mentions APIs they tend to mean Web API’s, an architectural approach that utilises HTTP as the application protocol of choice, with a number of characteristics like cacheability and statelessness (this is known as REST)  and a platform neutral payload format (like JSON or XML). You’re basically exposing your information assets in a standardised way making it very easy to connect to these services over the internet.

So far so good, everything looks sunny in this shiny new world of mobile clients and Web API’s, if it wasn’t for the law of conservation of misery. For every positive change there must be an equal negative change. You might also call it the double edged sword of technology, there’s always a catch somewhere. In the case of the Web API’s it’s a significant increase of headaches with regards to information security. The previous paragraph mentions that your information assets are exposed to the Internet as a set of Web API services. A Web API service looks like a regular web link (url), with the difference being that it returns information without any markup or formatting instructions. For instance will give you the geo location information for the Amazon Web Services data center where I host my blog (try it out, copy the url and paste it in your browser address bar to see the result). This means that everybody can access that service and get that information. Everybody means just that, everybody, there is no barrier to entry. That’s not a problem if you provide a free service like freegeoip that was used in the example but most businesses make money out of information and want to protect this valuable asset. A Web API that is open to anyone doesn’t help you with your entrepreneurial dreams.

How do you protect your Web API’s in this brave new world of independent single page applications? Your Web API requires protection on three levels: authentication, authorisation and input validation. This in itself doesn’t differ from a classic web application where the same protection is needed as well. The difference lies in the special circumstances that are introduced by dealing with Web API’s, the disconnected and independent nature of mobile clients and new security standards.

For the sake of this article lets assume that we’re dealing with a situation where the application consists of a mobile client, a Web API backend and a federated identity situation. The latter meaning that the application will be provided with an identity from a third party (for instance Facebook). That third party will have authenticated the user and is passing on the identity information. Authentication means that someone or something presents information (credentials) that can be verified by the backend system that provides the Web API. It could be the mobile client that presents these credentials or another Web API that is calling our Web API. As stated earlier the authentication takes place elsewhere at a so called Authorization Server (AS). The Authorization Server has actual information on the identity of the user and can verify whether the provided credentials are valid. If this is the case an identification (ID) token will be generated by the Authorization Server and passed to the client. When the client is requesting access to the Web API the ID token will be provided to the Web API which is called a Resource Server (RS) in this setup. The same mechanism can also be applied to a Web API calling another Web API. The same authentication mechanism can service both clients (browser, mobile) and Web API’s.

So far it’s been theory but how to make this practical? Many approaches have been tried in creating a federated identity approach for internet based systems and until only very recently these approaches were only partly successful. Either they were able to capture the Internet crowd or they were able to lure in the Enterprise folks but never both of them. You would have SAML for the corporates and something like OpenID for the Internet based systems. A new standard however changes all this and provides for the very first time a solution that is usable in the new mobile / Web API world and that is acceptable from a corporate perspective. This new standard is OpenID Connect. It’s the third version of OpenID and is has finally resolved a number of issues that have held back enterprise adoption of OpenID. It’s Internet orientation shows in its usage of HTTP as the carrier protocol and the usage of JSON based standards for transferring the required information (like JSON Web Tokens). It’s corporate orientation shows in the ability to convey information on how the authentication was carried out (like one-time-password, smartcard, fingerprint), when the user authenticated and information on the authenticated user (first name, last name, organization). The combination of features and protocol capabilities make for a clear winner when it concerns federated identity in a mobile /Web API world: OpenID Connect. Nothing else comes closer to a viable federated identity solution so for your own mobile applications you should seriously consider investing your time and money into OpenID Connect.

It’s quite likely that your Web API needs differentiated authentication, real life rarely is a “one size fits all” situation. You could have a Web API where some services are free to use, requiring no authentication, some services require username/password and your high end services might require a very strong form of authentication, perhaps biometry based (more common now that iPhones are shipping with fingerprint scanners). The way to resolve this is through a declarative authentication policy. An example of a declarative authentication policy:

Service Level Status
Rate None Active
Swap Medium Active
Admin High Active
BulkOrder High Blocked

In this schema, the service Rate doesn’t require any authentication and it can be called by any originator. The Swap service requires a Medium level authentication whereas Admin and BulkOrder require a High level authentication. The service BulkOrder has status “Blocked” and can’t be accessed in any shape or form. The “Blocked” status might be a temporary situation, every time a request comes in to the Web API the authentication policy will be checked so if a status changes from “Blocked” to “Active” it will allow traffic to previously blocked services. This doesn’t contradict the immutability aspect of declarative security. Immutability doesn’t mean nothing can ever change, it only means that the current state doesn’t change. If a new state is introduced then that is the new reality against which actions are verified and immutability is resumed. The authentication mechanisms themselves are defined separately to make it easier to change an authentication mechanism when needed. The authentication mechanism policy defines what constitutes a “Medium” or “High” level authentication. This looks as follows:

Level Mechanism
Medium Username/Password
Medium SMS One Time Password
High Fingerprint
High PKI smartcard

This approach requires an authentication framework that can identify and communicate differences in authentication mechanisms. Fortunately OpenID Connect provides the Authentication Method Reference (amr) field as the method for communicating what type of authentication was used at the end point, like “pwd” for username/password authentication or “fpt” for biometric fingerprint authentication. By using OpenID Connect and specifying what authentication level you require for which service you have a good mix of control and flexibility. If you want to change the type of authentication that you require for your Web API you only need to change the policy to make it effective.

In a run-time environment the aforementioned policies work as follows: a request is received by the Web API, it will do a lookup in the authentication policy file, determine which service is called (for instance Swap), retrieve the associated attributes (Authentication level=Medium, Status=Active), lookup the supported authentication mechanisms and verify whether all requirements have been met. If all requirements are met the Web API will provide the requested service, if not it will return an error.

By modelling all Web API services in this way you have a complete overview of all entry points into your systems. It also gives you full control in managing your Web API services from a security point of view, you can allow/block requests in a granular fashion. By making the security policy a part of your run-time environment you make sure no loopholes are inadvertently introduced. Every service needs to be defined in the authentication policy, if its not defined it doesn’t exist and can’t be called. This will ensure your authentication policy isn’t a paper tiger but an active component of your Web API security. If you keep the authentication policy at the right level of abstraction it should be low maintenance and provide a very clear added value to securing the Web API.

Next on the list is authorisation. The WebAPI requires authorisation because authentication will only give you a “ok” or “not ok” outcome, it’s not granular in the sense that some actions like read or create are ok but others like delete or update are not. Authentication is an all or nothing decision, there’s no grey area. Things are rarely this black and white and you could have various levels of subscribers to your services with for instance different prices for different services. In that case you want to be able to authorise users of your Web API for the services they are entitled to which is where authorisation comes into play.

Authorisation is a topic with a long history in security research. Much of the research effort has gone into dealing with the complexities of human interaction dealing with topics like delegation, limits and time/location constraints. These complexities are much less prevalent within Web API  authorisation nor should they be pursued. The focus should be on providing Create, Read, Update and Delete (CRUD) permissions. Complex topics like delegation, limits and time/location constraints need to be part of an authentication framework like OpenID Connect, as discussed above.

The authorisation mechanism has two parts: on the one hand the administration of which user has which permissions and on the other hand the definition (and grouping) of permissions. Both topics cover a well researched area, notably where its concerns Role Based Access Control (RBAC). In the RBAC methodology, permissions are grouped into roles and roles are assigned to users. One of the advantages of the RBAC approach is that changes in permissions (due to added, changed or deleted Web API services) don’t lead to a complete overhaul of the user administration because the association of user <–> role doesn’t change. It’s only the contents of the role that changes.  As such RBAC is an example of declarative authorisation where it concerns the administration aspect. Using a full RBAC system for Web API authorisation might be a bit of overkill but using an RBAC concept like the role abstraction is very useful. The important thing to remember when using RBAC concepts is that you don’t check for the role itself in your code but rather the capability that is required. So for instance the Rate service that was previously introduced requires a “read” capability to provide rate information. The Web API service will have the role of the user as part of the request context and when the Rate service is called the Web API authorization code will verify whether the role of the requesting user has a “read” capability. This way any number of roles can be created, changed or deleted without having any impact on the code itself. Let’s model the authorisations of a couple of the example Web API services as mentioned above in a declarative fashion.

Role Permission
Default ReadRate
Customer ReadRate, CreateSwap, ReadSwap, UpdateSwap
System Administrator CreateRate, ReadRate, UpdateRate, DeleteRate
Customer Administrator DeleteSwap

A client with the role “Customer” can access the Rate service because the Rate service will require a role with the capability (or permission) of ReadRate which is part of the “Customer” role. The way to determine which capability is required is by using the appropriate verbs in HTTP.

Action HTTP Verb

Calling the Rate service with a HTTP GET instruction means that the ReadRate method is called in which case the roles “Default” and “Customer” have the correct capabilities. Requesting the same Rate service with a HTTP POST invokes the CreateRate method which can only be done successfully by the “Customer” role.

The combination of the appropriate verb with the correct permission makes for an authorization solution that doesn’t require any software changes when the authorisation structure is changed. This can be verified by introducing a new service and assess the impact on the authorisation structure. Let’s introduce the service DeleteBulkOrder. It will be available to the “Customer Administrator” role only. In our table we will add DeleteBulkOrder to the role “Customer Administrator”. If the service “BulkOrder” is called with the required verb (HTTP DELETE) and is accessed with the proper role (Customer Administrator) its functionality will be available to the authorised user holding the role “Customer Administrator”. Likewise permissions can be added or removed from roles to make the associated services available or unavailable, all without changing a single line of code in the Web API and backend system itself.

Now that we have tackled authentication and authorisation its time for the last of our triad of secrets: input validation.  Insufficient or incorrect input validation is one of the biggest causes of security breaches as evidenced by the CWE/SANS TOP 25 Most Dangerous Software Errors list. This is an area where declarative security really shines and can provide an ironclad protection to your API. The trick with declarative input validation lies in specifying upfront what type of input you expect for which Web API Service. Lets take the Swap service as an example. The Swap service has three input parameters: “amount”, “uniqueswapindicator” and “timestamp”. The “amount” parameter states how many swaps need to be bought, the “uniqueswapindicator” states which swap needs to be bought and “timeframe” states how long the buy action remains valid. These parameters need to be analysed to determine what type of input we expect. The “amount” parameter expects a minimum of 1 and a maximum of 100 signifying the amount of swaps that can be bought with a single Web API call. This is a number (integer) type. The “uniqueswapindicator” is a parameter that contains a code indicating the type of swap that needs to be bought. The code contains letters and numbers (alphanumeric) and follows a pattern of two uppercase letters and 10 numbers (for example US0917971006). The last parameter is “timestamp” and it provides an ISO 8601 standard timestamp in the format YYYY-MM-DDThh:mm:ss.sTZD (for example 2015-06-26T19:20:30.45+01:00).

Modelling this is straightforward and looks like this:

Service Parameter Min Max Length Type
Swap amount 1 100 3 positive integer
Swap uniqueswapindicator 12 uniqueswapindicator
Swap timestamp 0 seconds 300 seconds 28 iso8601

This is a whitelist approach, the list specifically states what the Web API wants to receive and anything that doesn’t match the mandatory requirements will lead to a rejection of the request. The table contains a column “Service” which indicates with which WebAPI service the policy is associated. The “Parameter” column provides the match with the request parameter. The “Min” and “Max” columns indicate the minimum and maximum values of the parameter. The “Type” column provides a reference to a collection of content verification mechanisms, generally regular expression based, that are used to verify the content of the parameter. Taking the “amount” parameter as an example the validation policy declares that “amount” must be a value ranging from 1 to 100, with a maximum length of 3 positions and it needs to be a positive integer. The “uniqueswapindicator” has no min or max value because it is an identifier. It does however have a maximum length.  The “timestamp” parameter declaration states that timestamps may not have a date and time that is earlier than the current date and time and they have a maximum range of 5 minutes.

This declarative input validation policy can be transformed in any desired format to be used in the Web API logic, it could be JSON but it could also be native code for direct inclusion in the Web API run-time. Any incoming request will trigger the Web API to retrieve the input parameters for the requested service, do a lookup in the validation policy file, select the parameters that belong to the service, compare the received parameters with the retrieved parameters to verify whether the amount and name of the parameters are the same and subsequently do a content inspection of the parameters based on the criteria in the “Type” column of validation policy. If everything matches than the request can be processed by the backend, if there’s a matching failure the request is denied and generates an error. The Web API acts like a gate keeper protecting your backend logic from dealing with invalid or malicious requests.

Combining OpenID Connect, RBAC inspired Web API authorisation and declarative input validation makes for an iron clad protection of your Web API. Parts of the solution are available as commercial services so you can focus on providing your backend as a service to the world at large. A notable example of this is Amazon API Gateway where you can use OpenID Connect for authentication and provide authorisation based on access policies (not role based, put perhaps to come in the future). Declarative input validation is currently used in html5 and many javascript based form validation frameworks. It is a small step to adopt these practices in your Web API logic and benefit from the advantages of the many years of experience with this approach.

With these three secrets unveiled your Web API will benefit from increased security protection. Let’s prove the law of conservation of misery wrong.


By Meint Post, Managing Director of Mind IT, a consultancy based in The Netherlands and focused on IT Strategy, IT Architecture and IT Risk.

Meint is a nationally recognised IT and Security Architect, voted Top 3 IT Architect in The Netherlands by the Dutch Architecture Forum (NAF) in 2010. He has held a number of senior executive positions within the Rabobank International organisation over the past 15 years, bridging successfully from IT into the business. He brings a very deep knowledge of the internal dynamics of one of the largest banking players in Europe, coupled with a career-long interest in IT and Security.

The opinions expressed by guest bloggers are their views and do not necessarily reflect the opinions of Corix Partners.