web analytics

Archive for the ‘Application security’ Category

5 Golden Rules for Great Web API Design

http://www.toptal.com/#explore-competent-developers-today

The following article is a guest post from Toptal. Toptal is an elite network of freelancers that enables businesses to connect with the top 3% of software engineers and designers in the world.

Ever found yourself wondering “what were they thinking?” when integrating a web service via its API? If not, you’ve been far luckier than I have.

Any software developer knows how easy it is to let a project devolve into spaghetti code, and web APIs are no less prone to resulting in a tangled web. But it doesn’t need to be that way. In truth, it’s possible to design great web APIs that people will actually enjoy using, and that you’ll enjoy creating as well. But how? The answer to that question is what this post is all about.

Perspective

Most of the time when you’re building solutions, you’re designing for end users who are not programmers, or who are generally not technically sophisticated. You’re giving them a graphical interface and, if you’ve been doing your job right, you’ve gleaned a pretty good idea from them of what they need the interface to do.

But API development is different. You’re designing an interface for programmers, probably without even knowing who they are. And whoever they are, they will have the technical sophistication (or at least will think they have the technical sophistication) to point out every little flaw in your software. Your users are likely to be as critical of your API as you would be of theirs, and will thoroughly enjoy critiquing it.

And therein lies part of the irony, by the way. If anyone should understand how to make a web API that’s easy-to-use, it’s you. After all, you’re a software engineer just like the users of your API, so you share their perspective. Don’t you?

Well, while you certainly understand their perspective, you don’t necessarily share their perspective. When you’re developing or enhancing your API, you have the perspective of an API designer whereas they have the perspective of an API user.

API designers typically focus on questions like “What does this service need to do?” or “What does this service need to provide?”, while API users are focused on “How can I use this API to do what I need?”, or more accurately, “How can I spend the bare minimum of effort to get what I need out of this API?”.

These different questions lead to two vastly different perspectives. As a result, the necessary prerequisite to designing a great API is to shift your perspective from that of the API designer to that of the API user. In other words, continually ask yourself the questions you would naturally ask if you were your own user. Rather than thinking about what your API can do, think about the different ways it may need or want to be used and then focus on making those tasks as easy as possible for your API’s users.

While this may sound easy and obvious, it’s astounding how infrequently APIs appear to be designed this way. Think about the APIs you’ve encountered in your career. How frequently do they appear to have been designed with this perspective in mind? Web API design can be challenging.

So with that said, let’s proceed and talk about the 5 Golden Rules for Designing a Great Web API, namely:

  1. Documentation
  2. Stability and Consistency
  3. Flexibility
  4. Security
  5. Ease of Adoption

A diagram of users accessing a well-designed web API

Rule 1: Documentation

Documentation. Yes, I’m starting here.

Do you hate documentation? Well, I can empathize, but put on your “user perspective” hat and I’ll bet that the one thing you hate more than having to write documentation is having to try to use an undocumented API. I rest my case.

The bottom line is that, if you want anyone to use your API, documentation is essential. You’ve simply got to get this right. It’s the first thing users will see, so in some ways it’s like the gift wrap. Present well, and people are more likely to use your API and put up with any idiosyncrasies.

So how do we write good documentation?

The relatively easy part is documenting the API methods themselves; i.e., example requests and responses, along with descriptions of each of the elements in both. Fortunately, there are an increasing number of software tools that facilitate and simplify the task of generating documentation. Or you can write something yourself that introspects your API, endpoints, and functions, and generates the corresponding documentation for you.

But what separates great documentation from adequate documentation is the inclusion of usage examples and, ideally, tutorials. This is what helps the user understand your API and where to start. It orients them and helps them load your API into their brain.

For example, if the developers of Twilio were to list out every class, every method, and every possible response to their API, but didn’t bother to mention that you can send an SMS, track a call, or buy a phone number through their API, it would take a really long time for the API user to find that information and understand it cohesively. Can you imagine sorting through a giant tree of classes and methods without any insight into what they were used for, other than their name? Sounds terrible right? But that’s exactly what so many API providers do, thereby leaving their APIs opaque to anybody but themselves. The Rackspace CloudFiles developer and API guide is one such example; it’s difficult to get your bearings unless you already understand what they’re doing and what they’re providing.

So write concise tutorials that help get the developer up and running quickly, with at least a skeleton of what they’re trying to do, and then point them in the direction of the more detailed, fully-documented list of functionality so they can expand on what they have.

Once you’re done with your documentation, be sure to validate that it makes sense to people other than yourself. Send it out to other developers in your network, give them no instruction other than pointing them to the documentation, and ask them to follow a tutorial or build something really basic in about 15 minutes. If they can’t have a basic integration with your API in 15 minutes, you have more work to do.

For some noteworthy examples of excellent and detailed documentation, check out TwilioDjango, and MailChimp. None of these products are necessarily the best in their markets (although they are all good products), yet they do distinguish themeselves by providing some of the best documentation within their markets, which has certainly facilitated their wide acceptance and market share.

Rule 2: Stability and Consistency

If you’ve ever used Facebook’s API, you know how often they deprecate and completely rewrite their APIs. No matter how much you respect their hacker culture, or their product, their’s is not a developer-friendly perspective. The reason they are still successful is because they have a billion users, not because their API is great.

But you probably don’t have the luxury of such a mammoth user base and market share, so you’re going to need have a much less volatile API, keeping old versions running and supported for quite a long period of time. Maybe even years. So toward that end, here are some tips and tricks.

Let’s say, for example, that your API is accessible via the URL http://myapisite.com/api/widgets and provides its response in JSON format. While this may seem fine at first blush, what happens when you need to modify the format of the JSON response? Everyone that’s already integrated with you is going to break. Oops.

So do some planning ahead, and version your API from the outset, explicitly incorporating a version number into the URL (e.g., http://myapisite.com/api/widgets?version=1 or http://myapisite.com/api/widgets/v1) so that people can rely on version 1 working and can upgrade to any subsequent version when they’re ready to do so. If you need to phase out a prior version at some point, go ahead, but give plenty of notice and offer some sort of transition plan.

A good URL scheme will include major versions in the URL. Any change to the output format or supported data types should result in bumping up to a new major version. Generally, it’s acceptable to keep the same version if all you are doing is adding keys or nodes to your output, but to be on the safe side, any time the output changes, bump a version.

In addition to being stable over time, APIs need to be internally consistent. I’ve seen many APIs that change parameter names or methods of POSTing data, depending on the endpoint that is being used. Instead, you should handle common parameters globally within your API and use inheritance or a shared architecture to reuse the same naming conventions and data handling consistently throughout your API.

Finally, you need to record and publish a changelog to show differences between versions of your API so that users know exactly how to upgrade.

Rule 3: Flexibility

Garbage in, garbage out (GIGO) is a well known mantra to most programmers. As applied to web API design, this guiding principle tends to dictate a fairly rigid approach to request validation. Sounds great, right? No mess, no problem.

Yet as with everything, there needs to be some balance. As it is not possible to anticipate every way that users will want to employ your service, and since not every client platform is consistent (i.e., not every platform has very good JSON support, a decent OAuth library, etc.), it’s good to have at least some degree of flexibility or tolerance with regard to your input and output constraints.

For example, many APIs will support a variety of output formats, like JSON, YAML, XML, et. al., but will only support specifying the format in the URL itself. In the spirit of remaining flexible, you could allow this to also be specified in the URL (e.g., /api/v1/widgets.json), or you might also read and recognize an Accept: application/json HTTP header, or support a querystring variable such as ?format=JSON, and so on.

And while we’re at it, why not allow for the format specified to be case-insensitive, so the user could specify ?format=json as well? That’s a classic example of a way to alleviate unnecessary frustration for the user of your API.

Another example is allowing for different ways of inputting variables. So, just like you have a variety of output formats, allow for a variety of input formats as well (e.g., plain POST variables, JSON, XML, etc.). You should at least be supporting standard POST variables, and many modern applications support JSON as well, so those two are a good place to start.

The point here is that you shouldn’t assume that everyone shares your technical preferences. With a little research into how other APIs work, and through dialog with other developers, you can glean other valuable alternatives that are useful and include them in your API.

Rule 4: Security

Security is obviously one of the most important things to build into your web service, but so many developers make it ridiculously hard to use. As the API provider, you should be offering usable examples of how to authenticate and authorize when accessing your API. This should not be a difficult issue that an end user spends hours working on. Make it your goal that they either don’t have to write any code, or it takes them less than 5 minutes to write it.

For most APIs, I prefer a simple token-based authentication, where the token is a random hash assigned to the user and they can reset it at any point if it has been stolen. Allow the token to be passed in through POST or an HTTP header. For example, the user could (and should) send an SHA-1 token as a POST variable, or as a header in a format such as “Authorization: da39a3ee5e6b4b0d3255bfef95601890afd80709”.

Also, choose a secure token, not a short numeric identifier. Something irreversible is best. For example, it’s relatively simple to just generate out an SHA token during user creation and store it in the database. Then, you can simply query your database for any users matching that token. You could also do a token generated with a unique identifier and a salt value, something like SHA(User.ID + "abcd123"), and then query for any user that matches; e.g., where TokenFromPost = SHA(User.ID + "abcd123").

Another very good option is OAuth 2 + SSL. You should be using SSL anyway, but OAuth 2 is reasonably simple to implement on the server side, and libraries are available for many common programming languages.

If the API you have made is supposed to be accessible on a public website via JavaScript, you need to also make sure you validate a list of URLs per-account for the token. That way, nobody can go inspect the calls to your API, steal the token from your user, and go use it for themselves.

Here are some other important things to keep in mind:

  • Whitelisting Functionality. APIs generally allow you to do basic create, read, update, and delete operations on data. But you don’t want to allow these operations for every entity, so make sure each has a whitelist of allowable actions. Make sure, for example, that only authorized users can run commands like /user/delete/<id>. Similarly, all useful headers that are sent in the user’s request need to be validated against a whitelist as well. If you are allowing Content-type headers, verify that whatever the user sends in actually matches a whilelist of supported content types. If it doesn’t, then send back an error message such as a 406 Not Acceptable response. Whitelisting is important as a lot of APIs are automatically generated, or use a blacklist instead, which means you have to be explicit about what you don’t want. However, the golden rule of security is to start with absolutely nothing, and only explicitly allow what you do want.
  • Protect yourself against Cross-Site Request Forgery (CSRF). If you are allowing session or cookie authentication, you need to make sure that you’re protecting yourself from CSRF attacks. The Open Web Application Security Project (OWASP) provides useful guidance on ways to preclude these vulnerabilities.
  • Validate access to resources. In every request, you need to verify that a user is in fact allowed access to the specific item they are referencing. So, if you have an endpoint to view a user’s credit card details (e.g., /account/card/view/152423), be sure that the ID “152423” is referencing a resource that the user really is authorized to access.
  • Validate all input. All input from a user needs to be securely parsed, preferably using a well-known library if you are using complicated input like XML or JSON. Don’t build your own parser, or you’re in for a world of hurt.

Rule 5: Ease Of Adoption

This is really the most important rule in the bunch, and builds on all the others. As I mentioned during the documentation rule, try this out with people that are new to your API. Make sure that they can get up and running with at least a basic implementation of your API, even if it’s just following a tutorial, within a few minutes. I think 15 minutes is a good goal.

Here are some specific recommendations to ease and facilitate adoption of your API:

  • Make sure people can actually use your API and that it works the first time, every time. Have new people try to implement your API occasionally to verify that it’s not confusing in some way that you’ve become immune to.
  • Keep it simple. Don’t do any fancy authentication. Don’t do some crazy custom URL scheme. Don’t reinvent SOAP, or JSON, or REST, or anything. Use all the tools you can that have already been implemented and are widely accepted, so that developers only have to learn your API, not your API + 10 obscure new technologies.
  • Provide language-specific libraries to interface with your service. There are some nice tools to automatically generate a library for you, such as Alpaca or Apache Thrift. Currently Alpaca supports Node, PHP, Python, and Ruby. Thrift supports C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, JavaScript, Node.js, Smalltalk, OCaml, Delphi and more.
  • Simplify any necessary signup. If you are not developing an open source API, or if there is a signup process of any sort, make sure that upon signup, a user is very quickly directed to a tutorial. And make the signup process completely automated without any need for human interaction on your part.
  • Provide excellent support. A big barrier to adoption is lack of support. How will you handle and respond to a bug report? What about unclear documentation? An unsophisticated user? Forums, bug trackers, and email support are fantastic starts, but do make sure that when someone posts a bug, you really address it. Nobody wants to see a ghost town forum or a giant list of bugs that haven’t been addressed.

Web API Wrap-up

Web services and their APIs abound. Unfortunately, the vast majority are difficult to use. Reasons range from poor design, to lack of documentation, to volatility, to unresolved bugs, or, in some cases, all of the above.

Following the guidance in this post will help ensure that your web API is clean, well-documented, and easy-to-use. Such APIs are truly rare and are therefore that much more likely to be widely adopted and used.

Separation Anxiety: A Tutorial for Isolating Your System with Linux Namespaces

The following article is a guest post by Mahmud Ridwan, Technical Editor at Toptal. Toptal is an elite network of freelancers that enables businesses to connect with the top 3% of software engineers and designers in the world.

With the advent of tools like DockerLinux Containers, and others, it has become super easy to isolate Linux processes into their own little system environments. This makes it possible to run a whole range of applications on a single real Linux machine and ensure no two of them can interfere with each other, without having to resort to using virtual machines. These tools have been a huge boon to PaaS providers. But what exactly happens under the hood?

These tools rely on a number of features and components of the Linux kernel. Some of these features were introduced fairly recently, while others still require you to patch the kernel itself. But one of the key components, using Linux namespaces, has been a feature of Linux since version 2.6.24 was released in 2008.

Anyone familiar with chroot already has a basic idea of what Linux namespaces can do and how to use namespace generally. Just as chroot allows processes to see any arbitrary directory as the root of the system (independent of the rest of the processes), Linux namespaces allow other aspects of the operating system to be independently modified as well. This includes the process tree, networking interfaces, mount points, inter-process communication resources and more.

Why Use Namespaces for Process Isolation?

In a single-user computer, a single system environment may be fine. But on a server, where you want to run multiple services, it is essential to security and stability that the services are as isolated from each other as possible. Imagine a server running multiple services, one of which gets compromised by an intruder. In such a case, the intruder may be able to exploit that service and work his way to the other services, and may even be able compromise the entire server. Namespace isolation can provide a secure environment to eliminate this risk.

For example, using namespacing, it is possible to safely execute arbitrary or unknown programs on your server. Recently, there has been a growing number of programming contest and “hackathon” platforms, such as HackerRankTopCoderCodeforces, and many more. A lot of them utilize automated pipelines to run and validate programs that are submitted by the contestants. It is often impossible to know in advance the true nature of contestants’ programs, and some may even contain malicious elements. By running these programs namespaced in complete isolation from the rest of the system, the software can be tested and validated without putting the rest of the machine at risk. Similarly, online continuous integration services, such as Drone.io, automatically fetch your code repository and execute the test scripts on their own servers. Again, namespace isolation is what makes it possible to provide these services safely.

Namespacing tools like Docker also allow better control over processes’ use of system resources, making such tools extremely popular for use by PaaS providers. Services like Heroku and Google App Engine use such tools to isolate and run multiple web server applications on the same real hardware. These tools allow them to run each application (which may have been deployed by any of a number of different users) without worrying about one of them using too many system resources, or interfering and/or conflicting with other deployed services on the same machine. With such process isolation, it is even possible to have entirely different stacks of dependency softwares (and versions) for each isolated environment!

If you’ve used tools like Docker, you already know that these tools are capable of isolating processes in small “containers”. Running processes in Docker containers is like running them in virtual machines, only these containers are significantly lighter than virtual machines. A virtual machine typically emulates a hardware layer on top of your operating system, and then runs another operating system on top of that. This allows you to run processes inside a virtual machine, in complete isolation from your real operating system. But virtual machines are heavy! Docker containers, on the other hand, use some key features of your real operating system, including namespaces, and ensure a similar level of isolation, but without emulating the hardware and running yet another operating system on the same machine. This makes them very lightweight.

Process Namespace

Historically, the Linux kernel has maintained a single process tree. The tree contains a reference to every process currently running in a parent-child hierarchy. A process, given it has sufficient privileges and satisfies certain conditions, can inspect another process by attaching a tracer to it or may even be able to kill it.

With the introduction of Linux namespaces, it became possible to have multiple “nested” process trees. Each process tree can have an entirely isolated set of processes. This can ensure that processes belonging to one process tree cannot inspect or kill – in fact cannot even know of the existence of – processes in other sibling or parent process trees.

Every time a computer with Linux boots up, it starts with just one process, with process identifier (PID) 1. This process is the root of the process tree, and it initiates the rest of the system by performing the appropriate maintenance work and starting the correct daemons/services. All the other processes start below this process in the tree. The PID namespace allows one to spin off a new tree, with its own PID 1 process. The process that does this remains in the parent namespace, in the original tree, but makes the child the root of its own process tree.

With PID namespace isolation, processes in the child namespace have no way of knowing of the parent process’s existence. However, processes in the parent namespace have a complete view of processes in the child namespace, as if they were any other process in the parent namespace.

This namespace tutorial outlines the separation of various process trees using namespace systems in Linux.

It is possible to create a nested set of child namespaces: one process starts a child process in a new PID namespace, and that child process spawns yet another process in a new PID namespace, and so on.

With the introduction of PID namespaces, a single process can now have multiple PIDs associated with it, one for each namespace it falls under. In the Linux source code, we can see that a struct named pid, which used to keep track of just a single PID, now tracks multiple PIDs through the use of a struct named upid:

struct upid {
  int nr;                     // the PID value
  struct pid_namespace *ns;   // namespace where this PID is relevant
  // ...
};

struct pid {
  // ...
  int level;                  // number of upids
  struct upid numbers[0];     // array of upids
};

To create a new PID namespace, one must call the clone() system call with a special flag CLONE_NEWPID. (C provides a wrapper to expose this system call, and so do many other popular languages.) Whereas the other namespaces discussed below can also be created using the unshare() system call, a PID namespace can only be created at the time a new process is spawned using clone(). Once clone() is called with this flag, the new process immediately starts in a new PID namespace, under a new process tree. This can be demonstrated with a simple C program:

#define _GNU_SOURCE
#include <sched.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>

static char child_stack[1048576];

static int child_fn() {
  printf("PID: %ld\n", (long)getpid());
  return 0;
}

int main() {
  pid_t child_pid = clone(child_fn, child_stack+1048576, CLONE_NEWPID | SIGCHLD, NULL);
  printf("clone() = %ld\n", (long)child_pid);

  waitpid(child_pid, NULL, 0);
  return 0;
}

Compile and run this program with root privileges and you will notice an output that resembles this:

clone() = 5304
PID: 1

The PID, as printed from within the child_fn, will be 1.

Even though this namespace tutorial code above is not much longer than “Hello, world” in some languages, a lot has happened behind the scenes. The clone() function, as you would expect, has created a new process by cloning the current one and started execution at the beginning of the child_fn() function. However, while doing so, it detached the new process from the original process tree and created a separate process tree for the new process.

Try replacing the static int child_fn() function with the following, to print the parent PID from the isolated process’s perspective:

static int child_fn() {
  printf("Parent PID: %ld\n", (long)getppid());
  return 0;
}

Running the program this time yields the following output:

clone() = 11449
Parent PID: 0

Notice how the parent PID from the isolated process’s perspective is 0, indicating no parent. Try running the same program again, but this time, remove the CLONE_NEWPID flag from within the clone() function call:

pid_t child_pid = clone(child_fn, child_stack+1048576, SIGCHLD, NULL);

This time, you will notice that the parent PID is no longer 0:

clone() = 11561
Parent PID: 11560

However, this is just the first step in our tutorial. These processes still have unrestricted access to other common or shared resources. For example, the networking interface: if the child process created above were to listen on port 80, it would prevent every other process on the system from being able to listen on it.

Linux Network Namespace

This is where a network namespace becomes useful. A network namespace allows each of these processes to see an entirely different set of networking interfaces. Even the loopback interface is different for each network namespace.

Isolating a process into its own network namespace involves introducing another flag to the clone() function call: CLONE_NEWNET;

#define _GNU_SOURCE
#include <sched.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>


static char child_stack[1048576];

static int child_fn() {
  printf("New `net` Namespace:\n");
  system("ip link");
  printf("\n\n");
  return 0;
}

int main() {
  printf("Original `net` Namespace:\n");
  system("ip link");
  printf("\n\n");

  pid_t child_pid = clone(child_fn, child_stack+1048576, CLONE_NEWPID | CLONE_NEWNET | SIGCHLD, NULL);

  waitpid(child_pid, NULL, 0);
  return 0;
}

Output:

Original `net` Namespace:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
    link/ether 00:24:8c:a1:ac:e7 brd ff:ff:ff:ff:ff:ff


New `net` Namespace:
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN mode DEFAULT group default
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

What’s going on here? The physical ethernet device enp4s0 belongs to the global network namespace, as indicated by the “ip” tool run from this namespace. However, the physical interface is not available in the new network namespace. Moreover, the loopback device is active in the original network namespace, but is “down” in the child network namespace.

In order to provide a usable network interface in the child namespace, it is necessary to set up additional “virtual” network interfaces which span multiple namespaces. Once that is done, it is then possible to create Ethernet bridges, and even route packets between the namespaces. Finally, to make the whole thing work, a “routing process” must be running in the global network namespace to receive traffic from the physical interface, and route it through the appropriate virtual interfaces to to the correct child network namespaces. Maybe you can see why tools like Docker, which do all this heavy lifting for you, are so popular!

Linux network namespace is comprised of a routing process to multiple child net namespaces.

To do this by hand, you can create a pair of virtual Ethernet connections between a parent and a child namespace by running a single command from the parent namespace:

ip link add name veth0 type veth peer name veth1 netns <pid>

Here, <pid> should be replaced by the process ID of the process in the child namespace as observed by the parent. Running this command establishes a pipe-like connection between these two namespaces. The parent namespace retains the veth0 device, and passes the veth1 device to the child namespace. Anything that enters one of the ends, comes out through the other end, just as you would expect from a real Ethernet connection between two real nodes. Accordingly, both sides of this virtual Ethernet connection must be assigned IP addresses.

Mount Namespace

Linux also maintains a data structure for all the mountpoints of the system. It includes information like what disk partitions are mounted, where they are mounted, whether they are readonly, et cetera. With Linux namespaces, one can have this data structure cloned, so that processes under different namespaces can change the mountpoints without affecting each other.

Creating separate mount namespace has an effect similar to doing a chroot()chroot() is good, but it does not provide complete isolation, and its effects are restricted to the root mountpoint only. Creating a separate mount namespace allows each of these isolated processes to have a completely different view of the entire system’s mountpoint structure from the original one. This allows you to have a different root for each isolated process, as well as other mountpoints that are specific to those processes. Used with care per this tutorial, you can avoid exposing any information about the underlying system.

Learning how to use namespace correctly has multiple benefits as outlined in this namespace tutorial.

The clone() flag required to achieve this is CLONE_NEWNS:

clone(child_fn, child_stack+1048576, CLONE_NEWPID | CLONE_NEWNET | CLONE_NEWNS | SIGCHLD, NULL)

Initially, the child process sees the exact same mountpoints as its parent process would. However, being under a new mount namespace, the child process can mount or unmount whatever endpoints it wants to, and the change will affect neither its parent’s namespace, nor any other mount namespace in the entire system. For example, if the parent process has a particular disk partition mounted at root, the isolated process will see the exact same disk partition mounted at the root in the beginning. But the benefit of isolating the mount namespace is apparent when the isolated process tries to change the root partition to something else, as the change will only affect the isolated mount namespace.

Interestingly, this actually makes it a bad idea to spawn the target child process directly with the CLONE_NEWNSflag. A better approach is to start a special “init” process with the CLONE_NEWNS flag, have that “init” process change the “/”, “/proc”, “/dev” or other mountpoints as desired, and then start the target process. This is discussed in a little more detail near the end of this namespace tutorial.

Other Namespaces

There are other namespaces that these processes can be isolated into, namely user, IPC, and UTS. The user namespace allows a process to have root privileges within the namespace, without giving it that access to processes outside of the namespace. Isolating a process by the IPC namespace gives it its own interprocess communication resources, for example, System V IPC and POSIX messages. The UTS namespace isolates two specific identifiers of the system: nodename and domainname.

A quick example to show how UTS namespace is isolated is shown below:

#define _GNU_SOURCE
#include <sched.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/utsname.h>
#include <sys/wait.h>
#include <unistd.h>


static char child_stack[1048576];

static void print_nodename() {
  struct utsname utsname;
  uname(&utsname);
  printf("%s\n", utsname.nodename);
}

static int child_fn() {
  printf("New UTS namespace nodename: ");
  print_nodename();

  printf("Changing nodename inside new UTS namespace\n");
  sethostname("GLaDOS", 6);

  printf("New UTS namespace nodename: ");
  print_nodename();
  return 0;
}

int main() {
  printf("Original UTS namespace nodename: ");
  print_nodename();

  pid_t child_pid = clone(child_fn, child_stack+1048576, CLONE_NEWUTS | SIGCHLD, NULL);

  sleep(1);

  printf("Original UTS namespace nodename: ");
  print_nodename();

  waitpid(child_pid, NULL, 0);

  return 0;
}

This program yields the following output:

Original UTS namespace nodename: XT
New UTS namespace nodename: XT
Changing nodename inside new UTS namespace
New UTS namespace nodename: GLaDOS
Original UTS namespace nodename: XT

Here, child_fn() prints the nodename, changes it to something else, and prints it again. Naturally, the change happens only inside the new UTS namespace.

More information on what all of the namespaces provide and isolate can be found in the tutorial here

Cross-Namespace Communication

Often it is necessary to establish some sort of communication between the parent and the child namespace. This might be for doing configuration work within an isolated environment, or it can simply be to retain the ability to peek into the condition of that environment from outside. One way of doing that is to keep an SSH daemon running within that environment. You can have a separate SSH daemon inside each network namespace. However, having multiple SSH daemons running uses a lot of valuable resources like memory. This is where having a special “init” process proves to be a good idea again.

The “init” process can establish a communication channel between the parent namespace and the child namespace. This channel can be based on UNIX sockets or can even use TCP. To create a UNIX socket that spans two different mount namespaces, you need to first create the child process, then create the UNIX socket, and then isolate the child into a separate mount namespace. But how can we create the process first, and isolate it later? Linux provides unshare(). This special system call allows a process to isolate itself from the original namespace, instead of having the parent isolate the child in the first place. For example, the following code has the exact same effect as the code previously mentioned in the network namespace section:

#define _GNU_SOURCE
#include <sched.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>


static char child_stack[1048576];

static int child_fn() {
  // calling unshare() from inside the init process lets you create a new namespace after a new process has been spawned
  unshare(CLONE_NEWNET);

  printf("New `net` Namespace:\n");
  system("ip link");
  printf("\n\n");
  return 0;
}

int main() {
  printf("Original `net` Namespace:\n");
  system("ip link");
  printf("\n\n");

  pid_t child_pid = clone(child_fn, child_stack+1048576, CLONE_NEWPID | SIGCHLD, NULL);

  waitpid(child_pid, NULL, 0);
  return 0;
}

And since the “init” process is something you have devised, you can make it do all the necessary work first, and then isolate itself from the rest of the system before executing the target child.

Conclusion

This tutorial is just an overview of how to use namespaces in Linux. It should give you a basic idea of how a Linux developer might start to implement system isolation, an integral part of the architecture of tools like Docker or Linux Containers. In most cases, it would be best to simply use one of these existing tools, which are already well-known and tested. But in some cases, it might make sense to have your very own, customized process isolation mechanism, and in that case, this namespace tutorial will help you out tremendously.

There is a lot more going on under the hood than I’ve covered in this article, and there are more ways you might want to limit your target processes for added safety and isolation. But, hopefully, this can serve as a useful starting point for someone who is interested in knowing more about how namespace isolation with Linux really works.

Biometric Security – The Key To Passwordless Authentication Or A Fad?

The following article is a guest post by Nermin Hajdarbegovic, Technical Editor at Toptal. Toptal is an elite network of freelancers that enables businesses to connect with the top 3% of software engineers and designers in the world.

Passwordless authentication has been the Holy Grail of security for years, but progress has been painfully slow. This does not mean that huge strides have not been made, but unfortunately, most of these developments have been relegated to research labs or professional niches. Until a few years ago, the technology to implement passwordless authentication on a grand scale simply wasn’t available.

biometrics and biometric security

However, the industry juggernaut is slowly but surely changing this. There are a few technical, legal and even ethical considerations to take into account, but be as it may, biometric security and passwordless authentication is here to stay.

Biometrics are already changing the game, and they will continue to do so.

Why Go Paswordless In The First Place?

Since this is an engineering blog, I don’t feel the need to explain to a group of security-minded developers the upsides of fast logins. We need not look at the problem from a consumer perspective – all of us are compelled to use a myriad of online services and an ever increasing number of devices. This won’t change anytime soon, and if anything, the number of services and devices we will have to log into will keep increasing.

Of course, there are plenty of ways passwords are being dispensed with, including biometric authentication. From a user perspective, the use of Google, Microsoft and Facebook accounts to log into third-party services works, since the user can avoid password bloat and not have to create accounts for every service and device.

OAuth and OpenID have been used for years to consolidate digital identities, and the standards are employed by some of the biggest names in the tech industry.

Technically, this is not really a passwordless approach, but the average user might not see the distinction.

The pros and cons of using this approach are:

Pros:

  • Convenience
  • Easy to implement
  • Good security
  • Brand name peace of mind

Cons:

  • Dependency on a centralised service
  • All eggs in one basket – by compromising one account, an attacker can gain access to others
  • Potential security vulnerabilities, beyond your control, can be used against you
  • People may be reluctant to use such services due to privacy concerns

Much of this is true of alternative solutions, although it does not apply to security certificates which are usually relegated to business users rather than consumers. The pros outweigh the cons, hence we can already log into countless third-party services using our existing accounts.

How Can Biometrics and Biometric Security Help?

Using biometric authentication systems addresses many issues; there’s no reliance on centralised services, privacy is not a concern, and the user experience is not compromised – provided it’s done right. So, let’s take a look at the pros and cons.

Pros:

  • Fingerprint scanning is quick, cheap and relatively secure
  • Voice recognition is easy to use and difficult to manipulate
  • Iris scans are very secure and potentially more convenient than fingerprint scanning
  • Electrocardiogram technology offers “always on” authentication
  • All biometric security methods address privacy concerns while offering good security

Cons:

  • Biometrics are not suitable for all applications
  • Cost of deploying biometric security is often prohibitive
  • Support is limited to certain platforms and unavailable on most
  • Some technologies are still immature
  • Biometrics are not a silver bullet – security can still be compromised

Biometrics are not a new concept, or a new technology. Biometric security has been used in many industries for decades, and it’s been a staple of Hollywood script writers even longer. I am sure many readers had a chance to play around with facial recognition and fingerprint scanners on their notebooks years ago – I know I did, and I also know I was not impressed; most of these early solutions were cheap gimmicks.

However, we’ve come a long way since then. More processing power is available, along with vastly superior imaging sensors, and everything is backed by increasingly sophisticated software. This is why some of these technologies are making a comeback, which they’re doing with a vengeance.

Industry Gives Thumbs Up To Fingerprint Scanners

Apple’s Touch ID is probably the most recognisable fingerprint authentication solution on the market, but it’s by no means the only one. Apple opened Touch ID to third-party developers in iOS 8 and proceeded to integrate the technology in new iPhones and iPads, as well as its Apple Pay service.

This is why iOS has a clear lead over Android and other platforms; every new iPhone and iPad will ship with Touch ID until Cupertino comes up with something better.

fingerprint biometric security

This does not mean that Android should be written off because an increasing number of Android phones are shipping with fingerprint scanners. The first biometric authentication devices featured small scanners that required the user to swipe their finger over the scanner, but touch-scan units, similar to Apple’s, are becoming increasingly common. It is important to note that this feature is not reserved for expensive, flagship products – even some $200 phones marketed by Chinese vendors feature such scanners.

However, there is still a consideration; Google has not integrated a fingerprint scanner on any of its Nexus devices, although it is rumoured that it originally intended to include it on the Nexus 6 smartphone. In fact, Android Open Source Project (AOSP) provided evidence that fingerprint support was removed from the device. This is not good news for Android developers, as Google usually showcases new technology on Nexus devices and follows up with documentation and APIs, as was the case with NFC support on the Nexus S, or the barometer sensor on the Galaxy Nexus.

Still, this did not prevent vendors from using their own code, with a few types of scanners. But, this is bad news for developers whose hands are tied since there is no standard that would eliminate fragmentation and insure interoperability. Samsung tried to overcome the problem by allowing developers to play around with its Pass API, but this is still not an ideal solution. Motorola tried to do the same four years ago with its old Atrix devices.

A number of hardware manufacturers and developers also released SDKs enabling developers to integrate support for various fingerprint scanners, but the lack of a standardised environment that would reduce or eliminate fragmentation is still a big issue.

It may take a while before we see fingerprint scanners on most phones, but a lot of progress is being made. We went from no scanners on flagship phones to relatively reliable scanners on $200 phones in the space of a couple of years.

fingerprint scanners

But, how useful are they? Are they just gimmicks like first-generation fingerprint scanners on old notebooks?

The technology works, there is no doubt about that, but for the time being applications are limited. Software development has to catch up with hardware, we need more services that can use such solution, and we need more APIs and standards and guidelines from industry leaders (namely, Google). At this point, fingerprint scanners on many Android devices are gimmicks, nothing more.

Overall, fingerprint scanners are convenient, but they’re not an ideal solution. While every fingerprint is unique, there are still some security concerns. Many scanners can be tricked, although it is getting increasingly difficult to pull this off with a simple image. There are alternatives though, including 3D printing, and some morbid ways of doing this, as one security expert pointed out a couple of years ago.

Needless to say, you can’t use fingerprint readers with gloves, an injured thumb, or in other extreme situations. But, these are relatively minor drawbacks.

Microsoft Wants To Look You In The Eye

So, let’s sum up. Android and iOS can already use fingerprint scanners for biometric security, and they are currently underutilised. But what about desktop environments? We can unlock our phones and authenticate payments using biometrics, but we still work on desktops, so how about making them truly passwordless?

Microsoft recently announced Windows Hello and in case you missed it, check out the official Windows blog for a comprehensive overview of this initiative.

This is how Microsoft explains its vision for Windows Hello:

Instead of using a shared or shareable secret like a password, Windows 10 helps to securely authenticate to applications, websites and networks on your behalf—without sending up a password. Thus, there is no shared password stored on their servers for a hacker to potentially compromise.

Windows 10 will ask you to verify that you have possession of your device before it authenticates on your behalf, with a PIN or Windows Hello on devices with biometric sensors. Once authenticated with ‘Passport,’ you will be able to instantly access a growing set of websites and services across a range of industries – favorite commerce sites, email and social networking services, financial institutions, business networks and more.

Windows Hello is a biometric authentication system that will enable users to instantly access their Windows 10 devices, using fingerprint scanning, iris scanning or facial recognition. Microsoft says “plenty” of new Windows 10 devices will support Windows Hello, but, personally, I find one technique particularly interesting.

Iris scanning is one of the methods supported by Microsoft and it has a few benefits over the alternatives. It should be more reliable, and potentially more convenient, than fingerprint scanning. In case you were wondering, this won’t be handled by our webcams or phone cameras – Microsoft wants to use “a combination of special hardware and software” to make sure the system can’t be beaten.

iris scanning and biometrics

The iris scanner will rely on infrared technology (potentially, near-infrared). This means it will be able to operate in all lighting conditions and see your iris through glasses, even tinted glasses. Hardware designers won’t have to set aside a lot of room on a device to integrate the scanner; it could be integrated right next to the selfie cam on our mobiles, or as an addition to a standalone web cameras used on many office machines today. This means it could be easily retrofitted to existing desktop PCs.

Aside from infrared scanners, Microsoft will also use more traditional biometric security measures such as facial recognition, relying on Intel RealSense camera technology. This should help make Windows Hello more prolific, especially as users upgrade to new notebooks and hybrids based on Intel platforms.

On the mobile front, an iris scan offers several advantages over fingerprint authentication; it can work with gloves, iris injuries are a lot less common than thumb injuries, and it should be much more difficult to beat a consumer grade iris scanner than a fingerprint scanner.

There is another angle to Microsoft’s approach – the software giant won’t store users’ biometric data. The biometric signature will be secured locally on devices and shared with no one but the user. The signature will only be used to unlock the device and Passport, so it won’t be used to authenticate users over the network.

The jury is still out on Microsoft’s biometrics plans and we will have to wait for Windows 10 to see it in action.

What About Always-On Authentication?

While all these technologies might do a good job at replacing traditional passwords, there are emerging concepts that could give engineers more freedom. What if we could dispense with the process entirely, no passwords, no fingerprint scans – nothing?

“Always-on authentication” is the next frontier, and a number of ways of getting there have already been proposed. However, an important distinction needs to be made. Always-on authentication usually refers to machine-to-machine authentication, such as a system of “always-on” SSL authentication, SHH connections, NFC credentials and various networking technologies. These are usually developed to monitor and authenticate financial transactions, thus reducing the risk of online fraud.

There are relatively few solutions for always-on user authentication. One such example is Bionym’s Nymiwristband. It is a wearable device that looks a lot like your average fitness tracker, but it’s more clever than that.

always on authentication

Nymi scans the user’s unique electrocardiogram (ECG). This means that you only need to have the device on your wrist to provide always-on authentication. As long as your heart keeps beating, you’ll be logged in.

If you’re thinking of trying the same trick on the Apple Watch or Android Wear watches, hold your horses, we’re not there yet. The Nymi doesn’t merely track the user’s heart rate like a smartwatch, it actually analyses the shape of the user’s ECG wave, which takes a more sensitive sensor. Smartwatches sound like the ideal hardware platform for this application and, sooner or later, they will be able to do the same thing.

Imagine unlocking your phone, car, office and computer simply by being there and having a pulse? Logging into any account seamlessly, then paying for lunch, doing some shopping on the way home and maybe withdrawing cash from an ATM, all without having to juggle your groceries and credit cards. We’re not there, yet, but we are slowly getting there.

What Does All This Mean For Software Developers And Users?

For the time being, software developers can use off-the-shelf middleware and tokenization to deploy paswordless solutions. One such example is Passwordless, a token-based, open-source framework for Node.js and Express. In case you are interested in how it’s deployed, Mozilla has a comprehensive blog post that explains it.

It will take a while, but biometric building blocks are slowly falling into place. The current crop of passwordless technologies will be augmented, and eventually replaced by biometric authentication.

Many biometric security skeptics including many of my colleagues, don’t believe this will happen anytime soon, but I am an incorrigible optimist; I think passwordless security will be standard by the end of the decade, and this is why: If we merely observe one particular field, be it software or hardware, we will find countless problems with biometrics, many of which I’ve already outlined. However, if we take a few steps back and look at the big picture, if we take a look at new industry trends and the increasing emphasis on personal and corporate security, highly publicised security breaches, privacy concerns – we are bound to see things from a different perspective.

Even so, the elephant in the room isn’t privacy or B2B security, it’s mobile payments.

The volume of mobile transactions in the US is expected to more than double this year to $10bn. By 2018, Bloomberg expects the volume to reach $110bn. On a per-capita basis, the average American consumer will make about $30 in transactions this year, but by 2018 the number will go up to $330 per capita, for every man, woman and child. Assuming the same compound annual growth rate in 2019 and 2020, we could be looking at four digits per capita by 2021.

With that sort of money in play, what do you think?

The 5 Most Common Mistakes HTML5 Developers Make: A Beginner’s Guide

The following article is a guest post by Demir Selmanovic, the Lead Technical Editor at Toptal. Toptal is an elite network of freelancers that enables businesses to connect with the top 3% of software engineers and designers in the world.

It’s been over 20 years since Tim Berners-Lee and Robert Cailliau specified HTML, which became the standard markup language used to build the Internet. Ever since then, the HTML development community has begged for improvements to this language, but this cry was mostly heard by web browser developers who tried to ease the HTML issues of their colleagues. Unfortunately, this led to even more problems causing many cross-browser compatibility issues and duplication of development work. Over these 20 years, HTML was upgraded 4 times, while most of the browsers got double-digit numbers of major updates plus numerous small patches.

HTML5 was supposed to finally solve our problems and become one standard to rule them all (browsers). This was probably one of the most anticipated technologies since creation of the World Wide Web. Has it happened? Did we finally get one markup language that will be fully cross-browser compatible and will work on all desktop and mobile platforms giving us all of those features we were asking for? I dont know! Just few days ago (Sept. 16th 2014) we received one more call for review by W3C so the HTML5 specification is still incomplete.

Despite HTML5 issues and mistakes, is it the one language ring to rule them all?

Hopefully, when the specification is one day finalized, browsers will not already have significant obsolete code, and they will easily and properly implement great features like Web WorkersMultiple synchronized audio and video elements, and other HTML5 components that we’ve needed for a long time.

Give hasty developers an incomplete spec, and you’ll have a recipe for disaster.

We do however have thousands of companies that have based their businesses on HTML5 and are doing great. There are also many great HTML5-based applications and games used by millions of people, so success is obviously possible and HTML5 is, and will continue to be, used regardless on the status of its specification.

However, the recipe I mentioned can easily blow up in our faces, and thus I’ve emphasized some of the most basic HTML5 mistakes that can be avoided. Most of the mistakes listed below are consequence of incomplete or missing implementation of certain HTML5 elements in different browsers, and we should hope that in the near future they will become obsolete. Until this happens, I suggest reading the list and having it in mind when building your next HTML5 application whether you’re an HTML5 beginner or an experienced vet.

Common mistake #1: Trusting local storage

Let them eat cake! This approach has been a burden on developers for ages. Due to reasonably sensible fear of security breaches and protection of computers, in the “dark ages” when the Internet was feared by many, web applications were allowed to leave unreasonably small amounts of data on computers. True, there were things like user data that “great browser masters from Microsoft(r)” gave us, or things like Local Shared Objects in Flash, but this was far from perfect.

It is therefore reasonable that one of the first basic HTML5 features adopted by developers was Web Storage. However, be alert that Web Storage is not secure. It is better than using cookies because it will not be transmitted over the wire, but it is not encrypted. You should definitely never store security tokens there. Your security policy should never rely on data stored in Web Storage as a malicious user can easily modify her localStorage and sessionStorage values at any time.

To get more insight on Web Storage and how to use it, I suggest reading this post.

Common mistake #2: Expecting compatibility among browsers

HTML5 is much more than a simple markup language. It has matured enough to combine behavior together with layout, and you should consider HTML5 as extended HTML with advanced JavaScript on top. If you look at all the trouble we had before just to make a static combination of HTML+CSS look identical on all browsers, it is fair to assume that more complexity will just mean more effort assuring cross-browser compatibility.

HTML5 interpretation on different browsers is far from identical, just like the case was with JavaScript. All major players in the browser wars lended a hand in the HTML5 spec, so it’s fair to assume that deviations between browsers should reduce over time. But even now some browsers decided to fully ignore certain HTML5 elements making it very difficult to follow a baseline and common set of features. If we add more internet connected devices and platforms to the equation, the situation gets even more complicated, causing problems with HTML5.

For example Web Animations are great feature that is supported only by Chrome and Opera, while Web Notification feature that allows alerting the user outside the context of a web page of an occurrence, such as the delivery of email, is fully ignored by Internet Explorer.

To learn more about HTML5 features and support by different browsers check out the HTML5 guide at www.caniuse.com.

So the fact remains that even though HTML5 is new and well specified, we should expect a great deal of cross-browser compatibility issues and plan for them in advance. There are just too many gaps that browsers need to fill in, and it is fair to expect that they cannot overcome all of the differences between platforms well.

Common mistake #3: Assuming high performance

Regardless of the fact that HTML5 is still evolving, it is a very powerful technology that has many advantages over alternate platforms used before its existence. But, with great power comes great responsibility, especially for HTML5 beginners. HTML5 has been adopted by all major browsers on desktop and mobile platforms. Having this in mind, many development teams pick HTML5 as their preferred platform, hoping that their applications will run equally on all browsers. However, assuming sensible performance on both desktop and mobile platforms just because HTML5 specification says so, is not sensible. Lots of companies (khm! Facebook khm!) placed their bets on HTML5 for their mobile platform and suffered from HTML5 not working out as they planned.

Again, however, there are some great companies that rely heavily on HTML5. Just look at the numerous online game development studios that are doing amazing stuff while pushing HTML5 and browsers to their limits. Obviously, as long as you are aware of the performance issues and are working around these, you can be in a great place making amazing applications.

Common mistake #4: Limited accessibility

Web has become a very important part of our lives. Making applications accessible to people who rely upon assistive technology is important topic that is often put aside in software development. HTML5 tries to overcome this by implementing some of the advanced accessibility features. More than a few developers accepted this to be sufficient and haven’t really spent any time implementing additional accessibility options in their applications. Unfortunately, at this stage HTML5 has issues that prevent it from making your applications available to everyone and you should expect to invest additional time if you want to include a broader range of users.

You can check this place for more information about accessibility in HTML5 and the current state of the most common accessibility features.

Common mistake #5: Not using HTML5 enhancements

HTML5 has extended the standard HTML/XHTML set of tags significantly. In addition to new tags, we got quite a few new rules and behaviors. Too many developers picked up just a few enhancements and have skipped some of the very cool new features of HTML5.

One of the coolest things in HTML5 is client-side validation. This feature was probably one of the earliest elements of HTML5 that web browsers picked up. But, unfortunately, you can find more than a few developers who add novalidate attribute to their forms by default. The reasons for doing this arereasonable, and I’m quite sure we will have a debate about this one. Due to backward compatibility, many applications implemented custom JavaScript validators and having out-of-the-box validation done by browsers on top of that is inconvenient. However, it is not too difficult to assure that two validation methods will not collide, and standardizing user validation will ensure common experience while helping to resolve accessibility issues that I mentioned earlier.

Another great feature is related to the way user input is handled in HTML5. Before HTML5 came, we had to keep all of our form fields contained inside the <form></form> tag. New form attributes make it perfectly valid to do something like this:

<form action="demo_form.asp" id="form1">
  First name: <input type="text" name="fname"><br>
  <input type="submit" value="Submit">
</form>

Last name: <input type="text" name="lname" form="form1">

Even if lname is not inside the form, it will be posted together with fname.

For more information about new form attributes and enhancements, you can check the Mozilla Developer Network.

Wrap up

I understand that web developers are collateral damage in the browser wars as many of the above mistakes are a direct consequence of problematic HTML5 implementation in different browsers. However, it is still crucial that we avoid common HTML5 issues and spend some time understanding the new features of HTML5. Once we have it all under control, our applications will utilize great new enhancements and take the web to the next level.

Web developers are collateral damage in the browser wars.

Are We Creating An Insecure Internet of Things (IoT)? Security Challenges and Concerns

The Internet of Things (IoT) has been an industry buzzword for years, but sluggish development and limited commercialization have led some industry watchers to start calling it the “Internet of NoThings”.

Double puns aside, IoT development is in trouble. Aside from spawning geeky jokes unfit for most social occasions, the hype did not help; and, in fact, I believe it actually caused a lot more harm than good. There are a few problems with IoT, but all the positive coverage and baseless hype are one we could do without. The upside of generating more attention is clear: more investment, more VC funding, more consumer interest.

However, these come with an added level of scrutiny, which has made a number of shortcomings painfully obvious. After a couple of years of bullish forecasts and big promises, IoT security seems to be the biggest concern. The first few weeks of 2015 were not kind to this emerging industry, and most of the negative press revolved around security.

Was it justified? Was it just “fear, uncertainty and doubt” (FUD), brought about by years of hype? It was a bit of both; although some issues may have been overblown, the problems are very real, indeed.

From “Year Of IoT” To Annus Horribilis For IoT

Many commentators described 2015 as “the year of IoT,” but so far, it has been a year of bad press. Granted, there are still ten months to go, but negative reports keep piling on. Security firm Kaspersky recently ran a damning critique of IoT security challenges, with an unflattering headline, “Internet of Crappy Things”.

Kaspersky is no stranger to IoT criticism and controversy; the firm has been sounding alarm bells for a while, backing them up with examples of hacked smart homes, carwashes and even police surveillance systems. Whether a hacker wants to wash their ride free of charge, or stalk someone using their fitness tracker – IoT security flaws could make it possible.

Wind River published a white paper on IoT security in January 2015, and the report starts off with a sobering introduction. Titled Searching For The Silver Bullet, it summarizes the problem in just three paragraphs, which I will condense into a few points:

  • Security must be the foundational enabler for IoT.
  • There is currently no consensus on how to implement security in IoT on the device.
  • A prevalent, and unrealistic, expectation is that it is somehow possible to compress 25 years of security evolution into novel IoT devices.
  • There is no silver bullet that can effectively mitigate the threats.

However, there is some good news; the knowledge and experience are already here, but they have to be adapted to fit the unique constraints of IoT devices.

Unfortunately, this is where we as system security developers stumble upon another problem, a hardware problem.

U.S. Federal Trade Commission chairwoman, Edith Ramirez, addressed the Consumer Electronics Show in Las Vegas earlier this year, warning that embedding sensors into everyday devices, and letting them record what we do, could pose a massive security risk.

Ramirez outlined three key challenges for the future of IoT:

  • Ubiquitous data collection.
  • Potential for unexpected uses of consumer data.
  • Heightened security risks.

She urged companies to enhance privacy and built secure IoT devices by adopting a security-focused approach, reducing the amount of data collected by IoT devices, and increasing transparency and providing consumers with a choice to opt-out of data collection.

Ramirez went on to say that developers of IoT devices have not spent time thinking about how to secure their devices and services from cyberattacks.

“The small size and limited processing power of many connected devices could inhibit encryption and other robust security measures,” said Ramirez. “Moreover, some connected devices are low-cost and essentially disposable. If a vulnerability is discovered on that type of device, it may be difficult to update the software or apply a patch – or even to get news of a fix to consumers.”

While Ramirez is spot on in most respects, I should note that the Internet went through a similar phase two decades ago. There were a lot of security concerns, and the nineties saw the emergence of the internet-borne malware, DDoS attacks, sophisticated phishing and more. Even though Hollywood depicted a dystopian future in some films, we have ended up with kittens on social networks and a high-profile security breach here and there.

The Internet is still not secure, so we can’t expect IoT to be secure, either. However, security is constantly evolving to meet new challenges, we’ve seen it before, and we’ll see it again, with IoT and subsequent connected technologies.

IoT Hardware Is And Will Remain A Problem

Some of you will be thinking that the hardware issues mentioned by the FTC boss will be addressed; yes, some of them probably will.

As the IoT market grows, we will see more investment, and as hardware matures, we will get improved security. Chipmakers like Intel and ARM will be keen to offer better security with each new generation, since security could be a market differentiator, allowing them to grab more design wins and gain a bigger share.

Technology always advances, so why not? New manufacturing processes generally result in faster and more efficient processors, and sooner or later, the gap will close, thus providing developers with enough processing power to implement better security features. However, I am not so sure this is a realistic scenario.

insecure iot

First of all IoT chips won’t be big money-makers since they are tiny and usually based on outdated architectures. For example, the first-generation Intel Edison platform is based on Quark processors, which essentially use the same CPU instruction set and much of the design of the ancient Pentium P54C. However, the next-generation Edison microcomputer is based on a much faster processor, based on Atom Silvermont cores, which is in many Windows and Android tablets, today. (Intel shipped ~46m Bay Trail SoCs in 2014.)

On the face of it, we could end up with relatively modern 64-bit x86 CPU cores in IoT devices, but they won’t come cheap, they will still be substantially more complex than the smallest ARM cores, and therefore will need more battery power.

Cheap and disposable wearables, which appear to be the FTC’s biggest concern, won’t be powered by such chips, at least, not anytime soon. Consumers may end up with more powerful processors, such as Intel Atoms or ARMv8 chips, in some smart products, like smart refrigerators or washing machines with touchscreens, but they are impractical for disposable devices with no displays and with limited battery capacity.

Selling complete platforms, or reference designs for various IoT devices, could help chipmakers generate more revenue, while at the same time introduce more standardisation and security. The last thing the industry needs is more unstandardized devices and more fragmentation. This may sound like a logical and sound approach, since developers would end up with fewer platforms and more resources would be allocated for security, however, security breaches would also affect a bigger number of devices.

Money Is Pouring In, Analysts Remain Bullish, What Could Possibly Go Wrong?

One of the most common ways of tackling any problem in the tech industry is to simply throw money at it. So, let’s see where we stand right now in terms of funding rather than technology.

According to research firms IDC and Gartner, IoT will grow to such an extent that it will transform the data centre industry by the end of the decade. Gartner expects the IoT market will have 26 billion installed units by 2020, creating huge opportunities for all parties, from data centres and hardware makers, to developers and designers. IDC also expects the IoT industry to end up with “billions of devices and trillions of dollars” by the end of the decade.

Gartner’s latest comprehensive IoT forecast was published in May 2014 and it also includes a list of potential challenges, some of which I’ve already covered:

  • Security: Increased automation and digitization creates new security concerns.
  • Enterprise: Security issues could pose safety risks.
  • Consumer Privacy: Potential of privacy breaches.
  • Data: Lots of data will be generated, both for big data and personal data.
  • Storage Management: Industry needs to figure out what to do with the data in a cost-effective manner.
  • Server Technologies: More investment in servers will be necessary.
  • Data Centre Network: WAN links are optimised for human interface applications, IoT is expected to dramatically change patterns by transmitting data automatically.

All these points (and more) must be addressed sooner or later, often at a substantial cost. We are no longer talking about tiny IoT chips and cheap toys based on such chips, this is infrastructure. This is a lot of silicon in server CPUs, expensive DDR4 ECC RAM and even bigger SSDs, all housed in expensive servers, in even bigger data centres.

That’s just the tip of the iceberg; industry must tackle bandwidth concerns, data management and privacy policies, and security. So how much money does that leave for security, which is on top of Gartner’s list of IoT challenges?

A lot of money is already pouring into the industry, VCs are getting on board and the pace of investment appears to be picking up. There were also a number of acquisitions, often involving big players like Google, Qualcomm, Samsung, Gemalto, Intel and others. There is a list of IoT-related investments on Postscapes. The trouble with many of these investments, especially those coming from VCs, is that they tend to focus on “shiny” things, devices that can be marketed soon, with a potentially spectacular ROI. These investments don’t do much for security or infrastructure, which would basically have to trail IoT demand.

Big players will have to do the heavy lifting, not VC-backed startups and toymakers. Agile and innovative startups will certainly play a big role by boosting adoption and creating demand, but they can’t do everything.

So let’s think of it this way, even a small company can build a car, or tens of thousands of cars, but it can’t build highways, roads, petrol stations and refineries. That same small company can build a safe vehicle using off-the-shelf technology to meet basic road safety standards, but it couldn’t build a Segway-like vehicle that would meet the same safety standards, nor could anyone else. Automotive safety standards could never apply to such vechicles, we don’t see people commuting to work on Segways, so we cannot expect the traditional tech security standard to apply to underpowered IoT devices, either.

Having commuters checking their email or playing Candy Crush while riding their Segways through rush hour traffic does not sound very safe, does it? So why should we expect IoT devices to be as safe as other connected devices, with vastly more powerful hardware and mature operating systems? It may be a strange analogy, but the bottom line is that IoT devices cannot be expected to conform to the same security standards as fully fledged computers.

But Wait, There Weren’t That Many IoT Security Debacles…

True, we don’t see a lot of headlines about spectacular IoT security breaches, but let me put it this way: how many security related headlines did you see about Android Wear? One? Two? None? It is estimated there are fewer than a million Android Wear devices in the wild, so they’re simply not a prime target for hackers, or a subject for security researchers.

How many IoT devices do you own and use right now? How many does your business use? That’s where the “Internet of NoThings” joke comes from, most people don’t have any. The numbers keep going up, but the average consumer is not buying many, so where is that growth coming from? IoT devices are out there and the numbers are booming, driven by enterprise rather than the consumer market.

Verizon and ABI Research estimate that there were 1.2 billion different devices connected to the internet last year, but by 2020, they expect as many as 5.4 billion B2B IoT connections.

Smart wristbands, toasters and dog collars aren’t a huge concern from a security perspective, but Verizon’s latest IoT report focuses on something a bit more interesting: enterprise.

The number of Verizon’s machine-to-machine (M2M) connections in the manufacturing sector increased by 204 percent from 2013 to 2014, followed by finance and insurance, media and entertainment, healthcare, retail and transportation. The Verizon report includes a breakdown of IoT trends in various industries, so it offers insight into the business side of things.

The overall tone of the report is upbeat, but it also lists a number of security concerns. Verizon describes security breaches in the energy industry as “unthinkable,” describes IoT security as “paramount” in manufacturing, and let’s not even bring up potential risks in healthcare and transportation.

How And When Will We Get A Secure Internet of Things?

I will not try to offer a definitive answer on how IoT security challenges can be resolved, or when. The industry is still searching for answers and there is a long way to go. Recent studies indicate that the majority of currently available IoT devices have security vulnerabilities. HP found that as many 70 percent of IoT devices are vulnerable to attack.

While growth offers a lot of opportunities, IoT is still not mature, or secure. Adding millions of new devices, hardware endpoints, billions of lines of code, along with more infrastructure to cope with the load, creates a vast set of challenges, unmatched by anything we have experienced over the past two decades.

That is why I am not an optimist.

I don’t believe the industry can apply a lot of security lessons to IoT, at least not quickly enough, not over the next couple of years. In my mind, the Internet analogy is a fallacy, simply because the internet of the nineties did not have to deal with such vastly different types of hardware. Using encryption and wasting clock cycles on security is not a problem on big x86 CPUs or ARM SoCs, but it won’t work the same way with tiny IoT devices with a fraction of the processing power and a much different power consumption envelope.

More elaborate processors, with a biger die, need bigger packaging and have to dissipate more heat. They also need more power, which means bigger, heavier, more expensive batteries. To shave off weight and reduce bulk, manufacturers would have to resort to using exotic materials and production techniques. All of the above would entail more R&D spending, longer time-to-market and a bigger bill of materials. With substantially higher prices and a premium build, such devices could hardly be considered disposable.

the internet of things - iot

So what has to be done to make IoT secure? A lot. And everyone has a role to play, from tech giants to individual developers.

Let’s take a look at a few basic points, such as what can be done, and what is being done, to improve IoT security now:

  • Emphasise security from day one
  • Lifecycle, future-proofing, updates
  • Access control and device authentication
  • Know your enemy
  • Prepare for security breaches

A clear emphasis on security from day one is always a good thing, especially when dealing with immature technologies and underdeveloped markets. If you are planning to develop your own IoT infrastructure, or deploy an existing solution, do your research and stay as informed as possible. This may involve trade-offs, as you could be presented with a choice of boosting security at the cost of compromising the user experience, but it’s worth it as long as you strike the right balance. This cannot be done on the fly, you have to plan ahead, and plan well.

In the rush to bring new products and services to market, many companies are likely to overlook long-term support. It happens all the time, even in the big leagues, so we always end up with millions of unpatched and insecure computers and mobile devices. They are simply too old for most companies to bother with, and it is bound to be even worse with disposable IoT devices. Major phone vendors don’t update their software on 2-3 year old phones, so imagine what will happen with $20 IoT devices that might be on your network for years. Planned obsolescence may be a part of it, but the truth is that updating old devices does not make much financial sense for the manufacturer since they have better things to do with their resources. Secure IoT devices would either have to be secure by design and impervious from the start, or receive vital updates throughout their lifecycle, and I’m sure you will agree neither option sounds realistic, at least, not yet.

Implementing secure access control and device authentication sounds like an obvious thing to bring up, but we are not dealing with your average connected device here. Creating access controls, and authentication methods, that can be implemented on cheap and compact IoT devices without compromising the user experience, or adding unnecessary hardware, is harder than it seems. As I mentioned earlier, lack of processing power is another problem, as most advanced encryption techniques simply wouldn’t work very well, if at all. In a previous post, I looked at one alternative, outsourcing encryption via the blockchain technology; I am not referring to the Bitcoin blockchain, but similar crypto technologies that are already being studied by several industry leaders.

Si vis pacem, para bellum – if you want peace, prepare for war. It is vital to study threats and potential attackers before tackling IoT security. The threat level is not the same for all devices and there are countless considerations to take into account; would someone rather hack your daughter’s teddy bear, or something a bit more serious? It’s necessary to reduce data risk, keep as much personal data as possible from IoT devices, properly secure necessary data transfers, and so on. However, to do all this, you first need to study the threat.

If all else fails, at least be prepared for potential security breaches. Sooner or later they will happen, to you or someone else (well, preferably a competitor). Always have an exit strategy, a way of securing as much data as possible and rendering compromised data useless without wrecking your IoT infrastructure. It is also necessary to educate customers, employees and everyone else involved in the process about the risks of such breaches. Instruct them in what to do in case of a breach, and what to do to avoid one.

Of course, a good disclaimer and TOS will also help if you end up dealing with the worst-case scenario.

The post originaly appeared on the: Toptal Engineering Blog

 

The Role of WebRTC Technology In Online Security

WebRTC technology is rather new (spearheaded by Google in 2012 through the World Wide Web Consortium). It is a free project that provides browsers with Real-Time Communications. The technology is now widely used in live help customer support solutions, webinar platforms, chat rooms for dating, etc. But there are too little solutions for enhanced safety. It’s weird. Since this technology offers great opportunities in this field.

WebRTC opens great opportunities in secure communications online

In the case of WebRTC technology to create a communication channel between subscribers is used Peer to Peer method. At the same time, there is no data transfer to any server. It is a great advantage. This ensures the confidentiality of transmitted information.

The majority of modern communication services works through central server. It means that all history is stored on the server and third parties can get access to them.

Using WebRTC technology security provider Privatoria.net developed a solution for confidential communication online in 2013. The main difference is the absence of data transfer to the server. Only the subscribers’ web browsers are used.

 

Chat service provides users with an opportunity to exchange messages by establishing a direct connection between their browsers and uses Peer to Peer method to communicate online.

To create a communication channel between subscribers it is enough to get a one-time key, and pass it to the called subscriber by any means of communication available. When the communication session is over, the history is deleted and the browser is closed, all correspondence between the subscribers disappears from the system.

In such case, no one can gain access to the content of communications.

A user will benefit from:

  • Secure text messaging
  • Secure Voice Call
  • Secure Video Call
  • Secure Data Transfer

As WebRTC supports not all browsers, Secure Chat solution works only in Google Chrome, Opera and Mozilla. At now developers are working on beta application for Android which will be available in Google Play Market in the nearest month.

Therefore, it is good chance for all us today to communicate securely online.

The Basics of Secure Web Development

The internet has contributed a great deal to commerce around the world in the last decade, and of course with a whole new generation of people breaking into the online world we’re starting to see just what computers are capable of accomplishing. Particularly when there is malicious intent on the other side of that keyboard.

Hackers and crackers are one of the biggest threats the world has ever experienced; they can take your money, your products or even destroy your business from the inside out – and you’ll never see them do it, they might not leave a trace at all. That is the terrifying truth about the internet; in most cases those with the skills to take what they want have the skills to hide themselves from detection – so what can you do to stop them.

The easiest way of protecting your website is to ensure that your business have a securely developed website. Secure web development is a complex area, and most likely something that you will need the help of a professional in order to fully implement, but it is worth noting that there are three different levels of security to take into consideration for your website and thus three different levels that need to be securely developed in order to ensure the protection of your business.

Consider these levels almost like doors. If your website was a business property you would have three ways in to the top secret bits; a front door, a side door and a back door.

The front door is the user interface; the bit of the website that you yourself work with. Now; the web developer might have made you a big magnificent door, lovely and secure – the sort of user interface that lets you manage your stock, orders, customers and all of the individual aspects of your business effortlessly without giving anything up. However; if your passwords aren’t secure it’s the equivalent of putting a rubbish, rusty old lock on that lovely secure door – completely pointless and insecure. Easy access. This is the first place a hacker is going to look – why would they waste their time hunting down and trying to exploit tiny weaknesses in the back door if they could open the front door with one little shove?

Change your passwords regularly, select passwords that use upper case, lower case, numbers and punctuation. Do not use the same password for everything.

The side door is the programming. The code used to construct your website puts everything in place and says who can do what and when; everything is controlled with the code, so an opening here can cause big problems if a hacker finds it. There are a number of different potential security risks when it comes to the code; there are bugs, which are just general, little faults with the website that occur when something didn’t go quite as planned or something was missed in the development stage. They always happen and there isn’t a single piece of software that doesn’t have bugs, the secure ones are just those that resolve the bugs as soon as they’re found, which stops them from being exploited.

Another risk to that side door is an injection; sort of like a fake key. This is something some of the smarter hackers can accomplish by injecting their own instructions into your system when it sends off a command or query – they can intercept your command or query. For example; let’s say you perform a simple PHP query that will fetch the products from the database when your user selects a product category. Normally this sort of script would be accessed through the URL with a category id.

For example;

Let’s say you did a regular sql database select query looking for the category ID, your category information and URL command might look something like;

c.category_id = ‘ . $_GET[‘cat’] . ‘LIMIT 10’;

Now; obviously this example suggests that the clever programmer has included a limit to prevent what is going to happen next – but this won’t protect him. Poor clever programmer is about to be outsmarted.

First of all; the only thing the thing the hacker needs to do is find your product list page and look for everything, example;

Yourwebsite.com/productlist.php?cat=1 or 1=1-

Doesn’t look like anything special right? Well, with this alone the hacker can now see every single one of your products. Depending on how secure your website is this might let them find faults in the products, but it’s probably still not that dangerous right? Well, what if they did this;

/productlist.php?cat=1 or error!;#$

Yep – bet you’re horrified now, because this will typically reveal the DBMS version of the query, and sometimes expose your table and column names. Not dangerous enough for you? With the tables and columns are revealed the hacker can move on to attacking the user table, all thanks to exploiting a weakness in the products table.

/productlist.php?cat=1 and (select count(*) from users) > 0

Creating a new query inside the existing one means that they don’t need to verify the database connection; they’re using yours. They have access to your database not and their using it to find your user table, which can progress to finding how many users you have, and even finding the information within the user table. I’m quite sure I don’t need to specify why having access to your user database is such a bad thing.

So – if you want to avoid the injections you need to ensure that every bit of input data gets validated, reduce the amount of information shown when an error displays, really limit the database permissions to prevent php queries from being able to pull any more information than they absolutely need to and use parameters in your queries.

Finally – the back door. This is the server. You need to ensure that the server you use to host your information and website is secure. There have been a number of cases where highly secure websites were eventually hacked by first hacking a much lower security website that shared the host server. If you want to avoid this you can consider a dedicated server for your website, you should also consider keeping to companies hosting companies that offer support and security as part of the hosting package. Ask them what software their servers are running; this will give you an idea of how regularly they are updated – up to date servers are the most secure. Older software has had longer to be exploited and thus more of the weaknesses in these are already known to hackers.

 

 

Kate Critchlow is a young and enthusiastic writer with a particular interest for technology, covering everything from secure development to the latest gadget releases.