web analytics

Archive for the ‘Vulnerability Assessment’ Category

Best Practices for Deploying New Environments in the Cloud for the First Time

When organizations take their first steps to use public cloud services, they tend to look at a specific target.

My recommendation – think scale!

Plan a couple of steps ahead instead of looking at single server that serves just a few customers. Think about a large environment comprised of hundreds or thousands of servers, serving 10,000 customers concurrently.

Planning will allow you to manage the environment (infrastructure, information security and budget) when you do reach a scale of thousands of concurrent customers. The more we plan the deployment of new environments in advance, according to their business purposes and required resources required for each environment, it will be easier to plan to scale up, while maintaining high level security, budget and change management control and more.

In this three-part blog series, we will review some of the most important topics that will help avoid mistakes while building new cloud environments for the first time.

Resource allocation planning

The first step in resources allocation planning is to decide how to divide resources based on an organizational structure (sales, HR, infrastructure, etc.) or based on environments (production, Dev, testing, etc.)

In-order to avoid mixing resources (or access rights) between various environments, the best practice is to separate the environments as follows:

  • Share resource account (security products, auditing, billing management, etc.)
  • Development environment account (consider creating separate account for test environment purposes)
  • Production environment account

Separating different accounts or environments can be done using:

Tagging resources

Even when deploying a single server inside a network environment (AWS VPC, Azure Resource Group, GCP VPC), it is important to tag resources. This allows identifying which resources belong to which projects / departments / environments, for billing purposes.

Common tagging examples:

  • Project
  • Department
  • Environment (Prod, Dev, Test)

Beyond tagging, it is recommended to add a description to resources that support this kind of meta-data, in-order to locate resources by their target use.

Authentication, Authorization and Password Policy

In-order to ease the management of working with accounts in the cloud (and in the future, multiple accounts according to the various environments), the best practice is to follow the rules below:

  • Central authentication – In case the organization isn’t using Active Directory for central account management and access rights, the alternative is to use managed services such as AWS IAM, Google Cloud IAM, Azure AD, Oracle Cloud IAM, etc.

If managed IAM service is chosen, it is critical to set password policy according to the organization’s password policy (minimum password length, password complexity, password history, etc.)

  • If the central directory service is used by the organization, it is recommended to connect and sync the managed IAM service in the cloud to the organizational center directory service on premise (federated authentication).
  • It is crucial to protect privileged accounts in the cloud environment (such as AWS Root Account, Azure Global Admin, Azure Subscription Owner, GCP Project Owner, Oracle Cloud Service Administrator, etc.), among others, by limiting the use of privileged accounts to the minimum required, enforcing complex passwords, and password rotation every few months. This enables multi-factor authentication and auditing on privileged accounts, etc.
  • Access to resources should be defined according to the least privilege principle.
  • Access to resources should be set to groups instead of specific users.
  • Access to resources should be based on roles in AWS, Azure, GCP, Oracle Cloud, etc.

Audit Trail

It is important to enable auditing in all cloud environments, in-order to gain insights on access to resources, actions performed in the cloud environment and by whom. This is both security and change management reasons.

Common managed audit trail services:

  • AWS CloudTrail – It is recommended to enable auditing on all regions and forward the audit logs to a central S3 bucket in a central AWS account (which will be accessible only for a limited amount of user accounts).
  • Working with Azure, it is recommended to enable the use of Azure Monitor for the first phase, in-order to audit all access to resources and actions done inside the subscription. Later on, when the environment expands, you may consider using services such as Azure Security Center and Azure Sentinel for auditing purposes.
  • Google Cloud Logging – It is recommended to enable auditing on all GCP projects and forward the audit logs to the central GCP project (which will be accessible only for a limited amount of user accounts).
  • Oracle Cloud Infrastructure Audit service – It is recommended to enable auditing on all compartments and forward the audit logs to the Root compartment account (which will be accessible only for a limited amount of user accounts).

Budget Control

It is crucial to set a budget and budget alerts for any account in the cloud at in the early stages of working with in cloud environment. This is important in order to avoid scenarios in which high resource consumption happens due to human error, such as purchasing or consuming expensive resources, or of Denial of Wallet scenarios, where external attackers breach an organization’s cloud account and deploys servers for Bitcoin mining.

Common examples of budget control management for various cloud providers:

  • AWS Consolidated Billing – Configure central account among all the AWS account in the organization, in-order to forward billing data (which will be accessible only for a limited amount of user accounts).
  • GCP Cloud Billing Account – Central repository for storing all billing data from all GCP projects.
  • Azure Cost Management – An interface for configuring budget and budget alerts for all Azure subscriptions in the organization. It is possible to consolidate multiple Azure subscriptions to Management Groups in-order to centrally control budgets for all subscriptions.
  • Budget on Oracle Cloud Infrastructure – An interface for configuring budget and budget alerts for all compartments.

Secure access to cloud environments

In order to avoid inbound access from the Internet to resources in cloud environments (virtual servers, databases, storage, etc.), it is highly recommended to deploy a bastion host, which will be accessible from the Internet (SSH or RDP traffic) and will allow access and management of resources inside the cloud environment.

Common guidelines for deploying Bastion Host:

The more we expand the usage of cloud environments, we can consider deploying a VPN tunnel from the corporate network (Site-to-site VPN) or allow client VPN access from the Internet to the cloud environment (such as AWS Client VPN endpoint, Azure Point-to-Site VPN, Oracle Cloud SSL VPN).

Managing compute resources (Virtual Machines and Containers)

When selecting to deploy virtual machines in cloud environment, it is highly recommended to follow the following guidelines:

  • Choose an existing image from a pre-defined list in the cloud providers’ marketplace (operating system flavor, operating system build, and sometimes an image that includes additional software inside the base image).
  • Configure the image according to organizational or application demands.
  • Update all software versions inside the image.
  • Store an up-to-date version of the image (“Golden Image”) inside the central image repository in the cloud environment (for reuse).
  • In case the information inside the virtual machines is critical, consider using managed backup services (such as AWS Backup or Azure Backup).
  • When deploying Windows servers, it is crucial to set complex passwords for the local Administrator’s account, and when possible, join the Windows machine to the corporate domain.
  • When deploying Linux servers, it is crucial to use SSH Key authentication and store the private key(s) in a secure location.
  • Whenever possible, encrypt data at rest for all block volumes (the server’s hard drives / volumes).
  • It is highly recommended to connect the servers to a managed vulnerability assessment service, in order to detect software vulnerabilities (services such as Amazon Inspector or Azure Security Center).
  • It is highly recommended to connect the servers to a managed patch management service in-order to ease the work of patch management (services such as AWS Systems Manager Patch Manager, Azure Automation Update Management or Google OS Patch Management).

When selecting to deploy containers in the cloud environment, it is highly recommended to follow the following guidelines:

Storing sensitive information

It is highly recommended to avoid storing sensitive information, such as credentials, encryption keys, secrets, API keys, etc., in clear text inside virtual machines, containers, text files or on the local desktop.

Sensitive information should be stored inside managed vault services such as:

Object Storage

When using Object Storage, it is recommended to follow the following guidelines:

  • Avoid allowing public access to services such as Amazon S3, Azure Blob Storage, Google Cloud Storage, Oracle Cloud Object Storage, etc.
  • Enable audit access on Object Storage and store the access logs in a central account in the cloud environment (which will be accessible only for a limited amount of user accounts).
  • It is highly recommended to encrypt data at rest on all data inside Object Storage and when there is a business or regulatory requirement, and encrypt data using customer managed keys.
  • It is highly recommended to enforce HTTPS/TLS for access to object storage (users, computers and applications).
  • Avoid creating object storage bucket names with sensitive information, since object storage bucket names are unique and saved inside the DNS servers worldwide.

Networking

Advanced use of cloud environments

It allows consumption of services, rather than maintaining servers, operating systems, updates/patches, backup and availability, assuming managed services in cluster or replica mode is chosen.

  • Use Infrastructure as a Code (IoC) in-order to ease environment deployments, lower human errors and standardize deployment on multiple environments (Prod, Dev, Test).

Common Infrastructure as a Code alternatives:

Summary

To sum up:

Plan. Know what you need. Think scale.

If you use the best practices outlined here, taking off to the cloud for the first time will be an easier, safer and smoother ride then you might expect.

Additional references

Integrate security aspects in a DevOps process

A diagram of a common DevOps lifecycle:

The DevOps world meant to provide complementary solution for both quick development (such as Agile) and a solution for cloud environments, where IT personnel become integral part of the development process. In the DevOps world, managing large number of development environments manually is practically infeasible. Monitoring mixed environments become a complex solution and deploying large number of different builds is becoming extremely fast and sensitive to changes.

The idea behind any DevOps solution is to provide a solution for deploying an entire CI/CD process, which means supporting constant changes and immediate deployment of builds/versions.
For the security department, this kind of process is at first look a nightmare – dozen builds, partial tests, no human control for any change, etc.

For this reason, it is crucial for the security department to embrace DevOps attitude, which means, embedding security in any part of the development lifecycle, software deployment or environment change.

It is important to understand that there are no constant stages as we used to have in waterfall development lifecycle, and most of the stages are parallel – in the CI/CD world everything changes quickly, components can be part of different stages, and for this reason it is important to confer the processes, methods and tools in all developments and DevOps teams.

In-order to better understand how to embed security into the DevOps lifecycle, we need to review the different stages in the development lifecycle:

Planning phase

This stage in the development process is about gathering business requirements.

At this stage, it is important to embed the following aspects:

  • Gather information security requirements (such as authentication, authorization, auditing, encryptions, etc.)
  • Conduct threat modeling in-order to detect possible code weaknesses
  • Training / awareness programs for developers and DevOps personnel about secure coding

 

Creation / Code writing phase

This stage in the development process is about the code writing itself.

At this stage, it is important to embed the following aspects:

  • Connect the development environments (IDE) to a static code analysis products
  • Review the solution architecture by a security expert or a security champion on his behalf
  • Review open source components embedded inside the code

 

Verification / Testing phase

This stage in the development process is about testing, conducted mostly by QA personnel.

At this stage, it is important to embed the following aspects:

  • Run SAST (Static application security tools) on the code itself (pre-compiled stage)
  • Run DAST (Dynamic application security tools) on the binary code (post-compile stage)
  • Run IAST (Interactive application security tools) against the application itself
  • Run SCA (Software composition analysis) tools in-order to detect known vulnerabilities in open source components or 3rd party components

 

Software packaging and pre-production phase

This stage in the development process is about software packaging of the developed code before deployment/distribution phase.

At this stage, it is important to embed the following aspects:

  • Run IAST (Interactive application security tools) against the application itself
  • Run fuzzing tools in-order to detect buffer overflow vulnerabilities – this can be done automatically as part of the build environment by embedding security tests for functional testing / negative testing
  • Perform code signing to detect future changes (such as malwares)

 

Software packaging release phase

This stage is between the packaging and deployment stages.

At this stage, it is important to embed the following aspects:

  • Compare code signature with the original signature from the software packaging stage
  • Conduct integrity checks to the software package
  • Deploy the software package to a development environment and conduct automate or stress tests
  • Deploy the software package in a green/blue methodology for software quality and further security quality tests

 

Software deployment phase

At this stage, the software package (such as mobile application code, docker container, etc.) is moving to the deployment stage.

At this stage, it is important to embed the following aspects:

  • Review permissions on destination folder (in case of code deployment for web servers)
  • Review permissions for Docker registry
  • Review permissions for further services in a cloud environment (such as storage, database, application, etc.) and fine-tune the service role for running the code

 

Configure / operate / Tune phase

At this stage, the development is in the production phase and passes modifications (according to business requirements) and on-going maintenance.

At this stage, it is important to embed the following aspects:

  • Patch management processes or configuration management processes using tools such as Chef, Ansible, etc.
  • Scanning process for detecting vulnerabilities using vulnerability assessment tools
  • Deleting and re-deployment of vulnerable environments with an up-to-date environments (if possible)

 

On-going monitoring phase

At this stage, constant application monitoring is being conducted by the infrastructure or monitoring teams.

At this stage, it is important to embed the following aspects:

  • Run RASP (Runtime application self-production) tools
  • Implement defense at the application layer using WAF (Web application firewall) products
  • Implement products for defending the application from Botnet attacks
  • Implement products for defending the application from DoS / DDoS attacks
  • Conduct penetration testing
  • Implement monitoring solution using automated rules such as automated recovery of sensitive changes (tools such as GuardRails)

 

Security recommendations for developments based on CI/CD / DevOps process

  • It is highly recommended to perform on-going training for the development and DevOps teams on security aspects and secure development
  • It is highly recommended to nominate a security champion among the development and DevOps teams in-order to allow them to conduct threat modeling at early stages of the development lifecycle and in-order to embed security aspects as soon as possible in the development lifecycle
  • Use automated tools for deploying environments in a simple and standard form.
    Tools such as Puppet require root privileges for folders it has access to. In-order to lower the risk, it is recommended to enable folder access auditing.
  • Avoid storing passwords and access keys, hard-coded inside scripts and code.
  • It is highly recommended to store credentials (SSH keys, privileged credentials, API keys, etc.) in a vault (Solutions such as HashiCorp vault or CyberArk).
  • It is highly recommended to limit privilege access based on role (Role based access control) using least privileged.
  • It is recommended to perform network separation between production environment and Dev/Test environments.
  • Restrict all developer teams’ access to production environments, and allow only DevOps team’s access to production environments.
  • Enable auditing and access control for all development environments and identify access attempts anomalies (such as developers access attempt to a production environment)
  • Make sure sensitive data (such as customer data, credentials, etc.) doesn’t pass in clear text at transit. In-case there is a business requirement for passing sensitive data at transit, make sure the data is passed over encrypted protocols (such as SSH v2, TLS 1.2, etc.), while using strong cipher suites.
  • It is recommended to follow OWASP organization recommendations (such as OWASP Top10, OWASP ASVS, etc.)
  • When using Containers, it is recommended to use well-known and signed repositories.
  • When using Containers, it is recommended not to rely on open source libraries inside the containers, and to conduct scanning to detect vulnerable versions (including dependencies) during the build creation process.
  • When using Containers, it is recommended to perform hardening using guidelines such as CIS Docker Benchmark or CIS Kubernetes Benchmark.
  • It is recommended to deploy automated tools for on-going tasks, starting from build deployments, code review for detecting vulnerabilities in the code and open source code, and patch management processes that will be embedded inside the development and build process.
  • It is recommended to perform scanning to detect security weaknesses, using vulnerability management tools during the entire system lifetime.
  • It is recommended to deploy configuration management tools, in-order to detect and automatically remediate configuration anomalies from the original configuration.

 

Additional reading sources:

 

 

This article was written by Eyal Estrin, cloud security architect and Vitaly Unic, application security architect.

Top 10 Most Common Mistakes That Java Developers Make: A Java Beginner’s Tutorial

The following article is a guest post by Mikhail Selivanov, Freelance Java Developer at Toptal. Toptal is an elite network of freelancers that enables businesses to connect with the top 3% of software engineers and designers in the world.

Java is a programming language that was initially developed for interactive television, but over time it has become widespread over everywhere software can be used. Designed with the notion of object-oriented programming, abolishing the complexities of other languages such as C or C++, garbage collection, and an architecturally agnostic virtual machine, Java created a new way of programming. Moreover, it has a gentle learning curve and appears to successfully adhere to its own moto – “Write once, run everywhere”, which is almost always true; but Java problems are still present. I’ll be addressing ten Java problems that I think are the most common mistakes. In case you missed it, you may check out my post on The 5 Most Common Mistakes HTML5 Developers Make: A Beginner’s Guide here.

common java mistakes

Common Mistake #1: Neglecting Existing Libraries

It’s definitely a mistake for Java Developers to ignore the innumerable amount of libraries written in Java. Before reinventing the wheel, try to search for available libraries – many of them have been polished over the years of their existence and are free to use. These could be logging libraries, like logback and Log4j, or network related libraries, like Netty or Akka. Some of the libraries, such as Joda-Time, have become a de facto standard.

The following is a personal experience from one of my previous projects. The part of the code responsible for HTML escaping was written from scratch. It was working well for years, but eventually it encountered a user input which caused it to spin into an infinite loop. The user, finding the service to be unresponsive, attempted to retry with the same input. Eventually, all the CPUs on the server allocated for this application were being occupied by this infinite loop. If the author of this naive HTML escape tool had decided to use one of the well known libraries available for HTML escaping, such as HtmlEscapers from Google Guava, this probably wouldn’t have happened. At the very least, true for most popular libraries with a community behind it, the error would have been found and fixed earlier by the community for this library.

Common Mistake #2: Missing the ‘break’ Keyword in a Switch-Case Block

These Java issues can be very embarrassing, and sometimes remain undiscovered until run in production. Fallthrough behavior in switch statements is often useful; however, missing a “break” keyword when such behavior is not desired can lead to disastrous results. If you have forgotten to put a “break” in “case 0” in the code example below, the program will write “Zero” followed by “One”, since the control flow inside here will go through the entire “switch” statement until it reaches a “break”. For example:

public static void switchCasePrimer() {
    	int caseIndex = 0;
    	switch (caseIndex) {
        	case 0:
            	System.out.println("Zero");
        	case 1:
            	System.out.println("One");
            	break;
        	case 2:
            	System.out.println("Two");
            	break;
        	default:
            	System.out.println("Default");
    	}
}

In most cases, the cleaner solution would be to use polymorphism and move code with specific behaviors into separate classes. Java mistakes such as this one can be detected using static code analyzers, e.g. FindBugsand PMD.

Common Mistake #3: Forgetting to Free Resources

Every time a program opens a file or network connection, it is important for Java beginners to free the resource once you are done using it. Similar caution should be taken if any exception were to be thrown during operations on such resources. One could argue that the FileInputStream has a finalizer that invokes the close() method on a garbage collection event; however, since we can’t be sure when a garbage collection cycle will start, the input stream can consume computer resources for an indefinite period of time. In fact, there is a really useful and neat statement introduced in Java 7 particularly for this case, called try-with-resources:

private static void printFileJava7() throws IOException {
    try(FileInputStream input = new FileInputStream("file.txt")) {
        int data = input.read();
        while(data != -1){
            System.out.print((char) data);
            data = input.read();
        }
    }
}

This statement can be used with any object that implements the AutoClosable interface. It ensures that each resource is closed by the end of the statement.

Common Mistake #4: Memory Leaks

Java uses automatic memory management, and while it’s a relief to forget about allocating and freeing memory manually, it doesn’t mean that a beginning Java developer should not be aware of how memory is used in the application. Problems with memory allocations are still possible. As long as a program creates references to objects that are not needed anymore, it will not be freed. In a way, we can still call this memory leak. Memory leaks in Java can happen in various ways, but the most common reason is everlasting object references, because the garbage collector can’t remove objects from the heap while there are still references to them. One can create such a reference by defining class with a static field containing some collection of objects, and forgetting to set that static field to null after the collection is no longer needed. Static fields are considered GC roots and are never collected.

Another potential reason behind such memory leaks is a group of objects referencing each other, causing circular dependencies so that the garbage collector can’t decide whether these objects with cross-dependency references are needed or not. Another issue is leaks in non-heap memory when JNI is used.

The primitive leak example could look like the following:

final ScheduledExecutorService scheduledExecutorService = Executors.newScheduledThreadPool(1);
final Deque<BigDecimal> numbers = new LinkedBlockingDeque<>();
final BigDecimal divisor = new BigDecimal(51);

scheduledExecutorService.scheduleAtFixedRate(() -> {
	BigDecimal number = numbers.peekLast();
   	if (number != null && number.remainder(divisor).byteValue() == 0) {
     	System.out.println("Number: " + number);
		System.out.println("Deque size: " + numbers.size());
	}
}, 10, 10, TimeUnit.MILLISECONDS);

	scheduledExecutorService.scheduleAtFixedRate(() -> {
		numbers.add(new BigDecimal(System.currentTimeMillis()));
	}, 10, 10, TimeUnit.MILLISECONDS);

try {
	scheduledExecutorService.awaitTermination(1, TimeUnit.DAYS);
} catch (InterruptedException e) {
	e.printStackTrace();
}

This example creates two scheduled tasks. The first task takes the last number from a deque called “numbers” and prints the number and deque size in case the number is divisible by 51. The second task puts numbers into the deque. Both tasks are scheduled at a fixed rate, and run every 10 ms. If the code is executed, you’ll see that the size of the deque is permanently increasing. This will eventually cause the deque to be filled with objects consuming all available heap memory. To prevent this while preserving the semantics of this program, we can use a different method for taking numbers from the deque: “pollLast”. Contrary to the method “peekLast”, “pollLast” returns the element and removes it from the deque while “peekLast” only returns the last element.

To learn more about memory leaks in Java, please refer to our article that demystified this problem.

Common Mistake #5: Excessive Garbage Allocation

Excessive Garbage Allocation

Excessive garbage allocation may happen when the program creates a lot of short-lived objects. The garbage collector works continuously, removing unneeded objects from memory, which impacts applications’ performance in a negative way. One simple example:

String oneMillionHello = "";
for (int i = 0; i < 1000000; i++) {
    oneMillionHello = oneMillionHello + "Hello!";
}
System.out.println(oneMillionHello.substring(0, 6));

In Java, strings are immutable. So, on each iteration a new string is created. To address this we should use a mutable StringBuilder:

StringBuilder oneMillionHelloSB = new StringBuilder();
    for (int i = 0; i < 1000000; i++) {
        oneMillionHelloSB.append("Hello!");
    }
System.out.println(oneMillionHelloSB.toString().substring(0, 6));

While the first version requires quite a bit of time to execute, the version that uses StringBuilder produces a result in a significantly less amount of time.

Common Mistake #6: Using Null References without Need

Avoiding excessive use of null is a good practice. For example, it’s preferable to return empty arrays or collections from methods instead of nulls, since it can help prevent NullPointerException.

Consider the following method that traverses a collection obtained from another method, as shown below:

List<String> accountIds = person.getAccountIds();
for (String accountId : accountIds) {
    processAccount(accountId);
}

If getAccountIds() returns null when a person has no account, then NullPointerException will be raised. To fix this, a null-check will be needed. However, if instead of a null it returns an empty list, then NullPointerException is no longer a problem. Moreover, the code is cleaner since we don’t need to null-check the variable accountIds.

To deal with other cases when one wants to avoid nulls, different strategies may be used. One of these strategies is to use Optional type that can either be an empty object or a wrap of some value:

Optional<String> optionalString = Optional.ofNullable(nullableString);
if(optionalString.isPresent()) {
    System.out.println(optionalString.get());
}

In fact, Java 8 provides a more concise solution:

Optional<String> optionalString = Optional.ofNullable(nullableString);
optionalString.ifPresent(System.out::println);

Optional type has been a part of Java since version 8, but it has been well known for a long time in the world of functional programming. Prior to this, it was available in Google Guava for earlier versions of Java.

Common Mistake #7: Ignoring Exceptions

It is often tempting to leave exceptions unhandled. However, the best practice for beginner and experienced Java developers alike is to handle them. Exceptions are thrown on purpose, so in most cases we need to address the issues causing these exceptions. Do not overlook these events. If necessary, you can either rethrow it, show an error dialog to the user, or add a message to the log. At the very least, it should be explained why the exception has been left unhandled in order to let other developers know the reason.

selfie = person.shootASelfie();
try {
    selfie.show();
} catch (NullPointerException e) {
    // Maybe, invisible man. Who cares, anyway?
}

A clearer way of highlighting an exceptions’ insignificance is to encode this message into the exceptions’ variable name, like this:

try { selfie.delete(); } catch (NullPointerException unimportant) {  }

Common Mistake #8: Concurrent Modification Exception

This exception occurs when a collection is modified while iterating over it using methods other than those provided by the iterator object. For example, we have a list of hats and we want to remove all those that have ear flaps:

List<IHat> hats = new ArrayList<>();
hats.add(new Ushanka()); // that one has ear flaps
hats.add(new Fedora());
hats.add(new Sombrero());
for (IHat hat : hats) {
    if (hat.hasEarFlaps()) {
        hats.remove(hat);
    }
}

Concurrent Modification Exception

If we run this code, “ConcurrentModificationException” will be raised since the code modifies the collection while iterating it. The same exception may occur if one of the multiple threads working with the same list is trying to modify the collection while others iterate over it. Concurrent modification of collections in multiple threads is a natural thing, but should be treated with usual tools from the concurrent programming toolbox such as synchronization locks, special collections adopted for concurrent modification, etc. There are subtle differences to how this Java issue can be resolved in single threaded cases and multithreaded cases. Below is a brief discussion of some ways this can be handled in a single threaded scenario:

Collect objects and remove them in another loop

Collecting hats with ear flaps in a list to remove them later from within another loop is an obvious solution, but requires an additional collection for storing the hats to be removed:

List<IHat> hatsToRemove = new LinkedList<>();
for (IHat hat : hats) {
    if (hat.hasEarFlaps()) {
        hatsToRemove.add(hat);
    }
}
for (IHat hat : hatsToRemove) {
    hats.remove(hat);
}

Use Iterator.remove method

This approach is more concise, and it doesn’t need an additional collection to be created:

Iterator<IHat> hatIterator = hats.iterator();
while (hatIterator.hasNext()) {
    IHat hat = hatIterator.next();
    if (hat.hasEarFlaps()) {
        hatIterator.remove();
    }
}

Use ListIterator’s methods

Using the list iterator is appropriate when the modified collection implements List interface. Iterators that implement ListIterator interface support not only removal operations, but also add and set operations. ListIterator implements the Iterator interface so the example would look almost the same as the Iterator remove method. The only difference is the type of hat iterator, and the way we obtain that iterator with the “listIterator()” method. The snippet below shows how to replace each hat with ear flaps with sombreros using “ListIterator.remove” and “ListIterator.add” methods:

IHat sombrero = new Sombrero();
ListIterator<IHat> hatIterator = hats.listIterator();
while (hatIterator.hasNext()) {
    IHat hat = hatIterator.next();
    if (hat.hasEarFlaps()) {
        hatIterator.remove();
        hatIterator.add(sombrero);
    }
}

With ListIterator, the remove and add method calls can be replaced with a single call to set:

IHat sombrero = new Sombrero();
ListIterator<IHat> hatIterator = hats.listIterator();
while (hatIterator.hasNext()) {
    IHat hat = hatIterator.next();
    if (hat.hasEarFlaps()) {
        hatIterator.set(sombrero); // set instead of remove and add
    }
}

Use stream methods introduced in Java 8 With Java 8, programmers have the ability to transform a collection into a stream and filter that stream according to some criteria. Here is an example of how stream api could help us filter hats and avoid “ConcurrentModificationException”.

hats = hats.stream().filter((hat -> !hat.hasEarFlaps()))
        .collect(Collectors.toCollection(ArrayList::new));

The “Collectors.toCollection” method will create a new ArrayList with filtered hats. This can be a problem if the filtering condition were to be satisfied by a large number of items, resulting in a large ArrayList; thus, it should be use with care. Use List.removeIf method presented in Java 8 Another solution available in Java 8, and clearly the most concise, is the use of the “removeIf” method:

hats.removeIf(IHat::hasEarFlaps);

That’s it. Under the hood, it uses “Iterator.remove” to accomplish the behavior.

Use specialized collections

If at the very beginning we decided to use “CopyOnWriteArrayList” instead of “ArrayList”, then there would have been no problem at all, since “CopyOnWriteArrayList” provides modification methods (such as set, add, and remove) that don’t change the backing array of the collection, but rather create a new modified version of it. This allows iteration over the original version of the collection and modifications on it at the same time, without the risk of “ConcurrentModificationException”. The drawback of that collection is obvious – generation of a new collection with each modification.

There are other collections tuned for different cases, e.g. “CopyOnWriteSet” and “ConcurrentHashMap”.

Another possible mistake with concurrent collection modifications is to create a stream from a collection, and during the stream iteration, modify the backing collection. The general rule for streams is to avoid modification of the underlying collection during stream querying. The following example will show an incorrect way of handling a stream:

List<IHat> filteredHats = hats.stream().peek(hat -> {
    if (hat.hasEarFlaps()) {
        hats.remove(hat);
    }
}).collect(Collectors.toCollection(ArrayList::new));

The method peek gathers all the elements and performs the provided action on each one of them. Here, the action is attempting to remove elements from the underlying list, which is erroneous. To avoid this, try some of the methods described above.

Common Mistake #9: Breaking Contracts

Sometimes, code that is provided by the standard library or by a third-party vendor relies on rules that should be obeyed in order to make things work. For example, it could be hashCode and equals contract that when followed, makes working guaranteed for a set of collections from the Java collection framework, and for other classes that use hashCode and equals methods. Disobeying contracts isn’t the kind of error that always leads to exceptions or breaks code compilation; it’s more tricky, because sometimes it changes application behavior without any sign of danger. Erroneous code could slip into production release and cause a whole bunch of undesired effects. This can include bad UI behavior, wrong data reports, poor application performance, data loss, and more. Fortunately, these disastrous bugs don’t happen very often. I already mentioned the hashCode and equals contract. It is used in collections that rely on hashing and comparing objects, like HashMap and HashSet. Simply put, the contract contains two rules:

  • If two objects are equal, then their hash codes should be equal.
  • If two objects have the same hash code, then they may or may not be equal.

Breaking the contract’s first rule leads to problems while attempting to retrieve objects from a hashmap. The second rule signifies that objects with the same hash code aren’t necessarily equal. Let us examine the effects of breaking the first rule:

public static class Boat {
    private String name;

    Boat(String name) {
        this.name = name;
    }

    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (o == null || getClass() != o.getClass()) return false;

        Boat boat = (Boat) o;

        return !(name != null ? !name.equals(boat.name) : boat.name != null);
    }

    @Override
    public int hashCode() {
        return (int) (Math.random() * 5000);
    }
}

As you can see, class Boat has overridden equals and hashCode methods. However, it has broken the contract, because hashCode returns random values for the same object every time it’s called. The following code will most likely not find a boat named “Enterprise” in the hashset, despite the fact that we added that kind of boat earlier:

public static void main(String[] args) {
    Set<Boat> boats = new HashSet<>();
    boats.add(new Boat("Enterprise"));

    System.out.printf("We have a boat named 'Enterprise' : %b\n", boats.contains(new Boat("Enterprise")));
}

Another example of contract involves the finalize method. Here is a quote from the official java documentation describing its function:

”The general contract of finalize is that it is invoked if and when the JavaTM virtual machine has determined that there is no longer any means by which this object can be accessed by any thread (that has not yet died), except as a result of an action taken by the finalization of some other object or class which is ready to be finalized. The finalize method may take any action, including making this object available again to other threads; the usual purpose of finalize, however, is to perform cleanup actions before the object is irrevocably discarded. For example, the finalize method for an object that represents an input/output connection might perform explicit I/O transactions to break the connection before the object is permanently discarded.“

One could decide to use the finalize method for freeing resources like file handlers, but that would be a bad idea. This is because there’s no time guarantees on when finalize will be invoked, since it’s invoked during the garbage collection, and GC’s time is indeterminable.

Common Mistake #10: Using Raw Type Instead of a Parameterized One

Raw types, according to Java specifications, are types that are either not parametrized, or non-static members of class R that are not inherited from the superclass or superinterface of R. There were no alternatives to raw types until generic types were introduced in Java. It supports generic programming since version 1.5, and generics were undoubtedly a significant improvement. However, due to backward compatibility reasons, a pitfall has been left that could potentially break the type system. Let’s look at the following example:

List listOfNumbers = new ArrayList();
listOfNumbers.add(10);
listOfNumbers.add("Twenty");
listOfNumbers.forEach(n -> System.out.println((int) n * 2));

Here we have a list of numbers defined as a raw ArrayList. Since its type isn’t specified with type parameter, we can add any object into it. But in the last line we cast elements to int, double it, and print the doubled number to standard output. This code will compile without errors, but once running it will raise a runtime exception because we attempted to cast a string to an integer. Obviously, the type system is unable to help us write safe code if we hide necessary information from it. To fix the problem we need to specify the type of objects we’re going to store in the collection:

List<Integer> listOfNumbers = new ArrayList<>();

listOfNumbers.add(10);
listOfNumbers.add("Twenty");

listOfNumbers.forEach(n -> System.out.println((int) n * 2));

The only difference from the original is the line defining the collection:

List<Integer> listOfNumbers = new ArrayList<>();

The fixed code wouldn’t compile because we are trying to add a string into a collection that is expected to store integers only. The compiler will show an error and point at the line where we are trying to add the string “Twenty” to the list. It’s always a good idea to parametrize generic types. That way, the compiler is able to make all possible type checks, and the chances of runtime exceptions caused by type system inconsistencies are minimized.

Conclusion

Java as a platform simplifies many things in software development, relying both on sophisticated JVM and the language itself. However, its features, like removing manual memory management or decent OOP tools, don’t eliminate all the problems and issues a regular Java developer faces. As always, knowledge, practice and Java tutorials like this are the best means to avoid and address application errors – so know your libraries, read java, read JVM documentation, and write programs. Don’t forget about static code analyzers either, as they could point to the actual bugs and highlight potential bugs.

10 Most Common Web Security Vulnerabilities

For all too many companies, it’s not until after a breach has occurred that web security becomes a priority. During my years working as an IT Security professional, I have seen time and time again how obscure the world of IT Security is to so many of my fellow programmers.

An effective approach to IT security must, by definition, be proactive and defensive. Toward that end, this post is aimed at sparking a security mindset, hopefully injecting the reader with a healthy dose of paranoia.

In particular, this guide focuses on 10 common and significant web security pitfalls to be aware of, including recommendations on how they can be avoided. The focus is on the Top 10 Web Vulnerabilities identified by the Open Web Application Security Project (OWASP), an international, non-profit organization whose goal is to improve software security across the globe.

A little web security primer before we start – authentication and authorization

When speaking with other programmers and IT professionals, I often encounter confusion regarding the distinction between authorization and authentication. And of course, the fact the abbreviation auth is often used for both helps aggravate this common confusion. This confusion is so common that maybe this issue should be included in this post as “Common Web Vulnerability Zero”.

So before we proceed, let’s clearly the distinction between these two terms:

  • Authentication: Verifying that a person is (or at least appears to be) a specific user, since he/she has correctly provided their security credentials (password, answers to security questions, fingerprint scan, etc.).
  • Authorization: Confirming that a particular user has access to a specific resource or is granted permission to perform a particular action.

Stated another way, authentication is knowing who an entity is, while authorization is knowing what a given entity can do.

Common Mistake #1: Injection flaws

Injection flaws result from a classic failure to filter untrusted input. It can happen when you pass unfiltered data to the SQL server (SQL injection), to the browser (XSS – we’ll talk about this later), to the LDAP server (LDAP injection), or anywhere else. The problem here is that the attacker can inject commands to these entities, resulting in loss of data and hijacking clients’ browsers.

Anything that your application receives from untrusted sources must be filtered, preferably according to a whitelist. You should almost never use a blacklist, as getting that right is very hard and usually easy to bypass. Antivirus software products typically provide stellar examples of failing blacklists. Pattern matching does not work.

Prevention: The good news is that protecting against injection is “simply” a matter of filtering your input properly and thinking about whether an input can be trusted. But the bad news is that all input needs to be properly filtered, unless it can unquestionably be trusted (but the saying “never say never” does come to mind here).

In a system with 1,000 inputs, for example, successfully filtering 999 of them is not sufficient, as this still leaves one field that can serve as the Achilles heal to bring down your system. And you might think that putting an SQL query result into another query is a good idea, as the database is trusted, but if the perimeter is not, the input comes indirectly from guys with malintent. This is called Second Order SQL Injection in case you’re interested.

Since filtering is pretty hard to do right (like crypto), what I usually advise is to rely on your framework’s filtering functions: they are proven to work and are thoroughly scrutinized. If you do not use frameworks, you really need to think hard about whether not using them really makes sense in your environment. 99% of the time it does not.

Common Mistake #2: Broken Authentication

This is a collection of multiple problems that might occur during broken authentication, but they don’t all stem from the same root cause.

Assuming that anyone still wants to roll their own authentication code in 2014 (what are you thinking??), I advise against it. It is extremely hard to get right, and there are a myriad of possible pitfalls, just to mention a few:

  1. The URL might contain the session id and leak it in the referer header to someone else.
  2. The passwords might not be encrypted either in storage or transit.
  3. The session ids might be predictable, thus gaining access is trivial.
  4. Session fixation might be possible.
  5. Session hijacking might be possible, timeouts not implemented right or using HTTP (no SSL), etc…

Prevention: The most straightforward way to avoid this web security vulnerability is to use a framework. You might be able to implement this correctly, but the former is much easier. In case you do want to roll your own code, be extremely paranoid and educate yourself on what the pitfalls are. There are quite a few.

Common Mistake #3: Cross Site Scripting (XSS)

This is a fairly widespread input sanitization failure (essentially a special case of common mistake #1). An attacker gives your web application JavaScript tags on input. When this input is returned to the user unsanitized, the user’s browser will execute it. It can be as simple as crafting a link and persuading a user to click it, or it can be something much more sinister. On page load the script runs and, for example, can be used to post your cookies to the attacker.

Prevention: There’s a simple web security solution: don’t return HTML tags to the client. This has the added benefit of defending against HTML injection, a similar attack whereby the attacker injects plain HTML content (such as images or loud invisible flash players) – not high-impact but surely annoying (“please make it stop!”). Usually, the workaround is simply converting all HTML entities, so that <script> is returned as &lt;script&gt;. The other often employed method of sanitization is using regular expressions to strip away HTML tags using regular expressions on < and >, but this is dangerous as a lot of browsers will interpret severely broken HTML just fine. Better to convert all characters to their escaped counterparts.

Common Mistake #4: Insecure Direct Object References

This is a classic case of trusting user input and paying the price in a resulting security vulnerability. A direct object reference means that an internal object such as a file or database key is exposed to the user. The problem with this is that the attacker can provide this reference and, if authorization is either not enforced (or is broken), the attacker can access or do things that they should be precluded from.

For example, the code has a download.php module that reads and lets the user download files, using a CGI parameter to specify the file name (e.g., download.php?file=something.txt). Either by mistake or due to laziness, the developer omitted authorization from the code. The attacker can now use this to download any system files that the user running PHP has access to, like the application code itself or other data left lying around on the server, like backups. Uh-oh.

Another common vulnerability example is a password reset function that relies on user input to determine whose password we’re resetting. After clicking the valid URL, an attacker can just modify the username field in the URL to say something like “admin”.

Incidentally, both of these examples are things I myself have seen appearing often “in the wild”.

Prevention: Perform user authorization properly and consistently, and whitelist the choices. More often than not though, the whole problem can be avoided by storing data internally and not relying on it being passed from the client via CGI parameters. Session variables in most frameworks are well suited for this purpose.

Common Mistake #5: Security misconfiguration

In my experience, web servers and applications that have been misconfigured are way more common than those that have been configured properly. Perhaps this because there is no shortage of ways to screw up. Some examples:

  1. Running the application with debug enabled in production.
  2. Having directory listing enabled on the server, which leaks valuable information.
  3. Running outdated software (think WordPress plugins, old PhpMyAdmin).
  4. Having unnecessary services running on the machine.
  5. Not changing default keys and passwords. (Happens way more frequently than you’d believe!)
  6. Revealing error handling information to the attackers, such as stack traces.

Prevention: Have a good (preferably automated) “build and deploy” process, which can run tests on deploy. The poor man’s security misconfiguration solution is post-commit hooks, to prevent the code from going out with default passwords and/or development stuff built in.

Common Mistake #6: Sensitive data exposure

This web security vulnerability is about crypto and resource protection. Sensitive data should be encrypted at all times, including in transit and at rest. No exceptions. Credit card information and user passwords should never travel or be stored unencrypted, and passwords should always be hashed. Obviously the crypto/hashing algorithm must not be a weak one – when in doubt, use AES (256 bits and up) and RSA (2048 bits and up).

And while it goes without saying that session IDs and sensitive data should not be traveling in the URLs and sensitive cookies should have the secure flag on, this is very important and cannot be over-emphasized.

Prevention:

  • In transit: Use HTTPS with a proper certificate and PFS (Perfect Forward Secrecy). Do not accept anything over non-HTTPS connections. Have the secure flag on cookies.
  • In storage: This is harder. First and foremost, you need to lower your exposure. If you don’t need sensitive data, shred it. Data you don’t have can’t be stolen. Do not store credit card information ever, as you probably don’t want to have to deal with being PCI compliant. Sign up with a payment processor such as Stripe or Braintree. Second, if you have sensitive data that you actually do need, store it encrypted and make sure all passwords are hashed. For hashing, use of bcrypt is recommended. If you don’t use bcrypt, educate yourself on salting and rainbow tables.

And at the risk of stating the obvious, do not store the encryption keys next to the protected data. That’s like storing your bike with a lock that has the key in it. Protect your backups with encryption and keep your keys very private. And of course, don’t lose the keys!

Common Mistake #7: Missing function level access control

This is simply an authorization failure. It means that when a function is called on the server, proper authorization was not performed. A lot of times, developers rely on the fact that the server side generated the UI and they think that the functionality that is not supplied by the server cannot be accessed by the client. It is not as simple as that, as an attacker can always forge requests to the “hidden” functionality and will not be deterred by the fact that the UI doesn’t make this functionality easily accessible. Imagine there’s an /adminpanel, and the button is only present in the UI if the user is actually an admin. Nothing keeps an attacker from discovering this functionality and misusing it if authorization is missing.

Prevention: On the server side, authorization must always be done. Yes, always. No exceptions or vulnerabilities will result in serious problems.

Common Mistake #8: Cross Site Request Forgery (CSRF)

This is a nice example of a confused deputy attack whereby the browser is fooled by some other party into misusing its authority. A 3rd party site, for example, can make the user’s browser misuse it’s authority to do something for the attacker.

In the case of CSRF, a 3rd party site issues requests to the target site (e.g., your bank) using your browser with your cookies / session. If you are logged in on one tab on your bank’s homepage, for example, and they are vulnerable to this attack, another tab can make your browser misuse its credentials on the attacker’s behalf, resulting in the confused deputy problem. The deputy is the browser that misuses its authority (session cookies) to do something the attacker instructs it to do.

Consider this example:

Attacker Alice wants to lighten target Todd’s wallet by transfering some of his money to her. Todd’s bank is vulnerable to CSRF. To send money, Todd has to access the following URL:

http://example.com/app/transferFunds?amount=1500&destinationAccount=4673243243

After this URL is opened, a success page is presented to Todd, and the transfer is done. Alice also knows, that Todd frequently visits a site under her control at blog.aliceisawesome.com, where she places the following snippet:

<img src="https://example.com/app/transferFunds?amount=1500&destinationAccount=4673243243" width="0" height="0" />

Upon visiting Alice’s website, Todd’s browser thinks that Alice links to an image, and automatically issues an HTTP GET request to fetch the “picture”, but this actually instructs Todd’s bank to transfer $1500 to Alice.

Incidentally, in addition to demonstrating the CSRF vulnerability, this example also demonstrates altering the server state with an idempotent HTTP GET request which is itself a serious vulnerability. HTTP GET requests must be idempotent (safe), meaning that they cannot alter the resource which is accessed. Never, ever, ever use idempotent methods to change the server state.

Fun fact: CSRF is also the method people used for cookie-stuffing in the past until affiliates got wiser.

Prevention: Store a secret token in a hidden form field which is inaccessible from the 3rd party site. You of course always have to verify this hidden field. Some sites ask for your password as well when modifying sensitive settings (like your password reminder email, for example), although I’d suspect this is there to prevent the misuse of your abandoned sessions (in an internet cafe for example).

Common Mistake #9: Using components with known vulnerabilities

The title says it all. I’d again classify this as more of a maintenance/deployment issue. Before incorporating new code, do some research, possibly some auditing. Using code that you got from a random person on GitHub or some forum might be very convenient, but is not without risk of serious web security vulnerability.

I have seen many instances, for example, where sites got owned (i.e., where an outsider gains administrative access to a system), not because the programmers were stupid, but because a 3rd party software remained unpatched for years in production. This is happening all the time with WordPress plugins for example. If you think they will not find your hidden phpmyadmin installation, let me introduce you to dirbuster.

The lesson here is that software development does not end when the application is deployed. There has to be documentation, tests, and plans on how to maintain and keep it updated, especially if it contains 3rd party or open source components.

Prevention:

  • Exercise caution. Beyond obviously using caution when using such components, do not be a copy-paste coder. Carefully inspect the piece of code you are about to put into your software, as it might be broken beyond repair (or in some cases, intentionally malicious).
  • Stay up-to-date. Make sure you are using the latest versions of everything that you trust, and have a plan to update them regularly. At least subscribe to a newsletter of new security vulnerabilities regarding the product.

Common Mistake #10: Unvalidated redirects and forwards

This is once again an input filtering issue. Suppose that the target site has a redirect.php module that takes a URL as a GET parameter. Manipulating the parameter can create a URL on targetsite.com that redirects the browser to malwareinstall.com. When the user sees the link, they will see targetsite.com/blahblahblahwhich the user thinks is trusted and is safe to click. Little do they know that this will actually transfer them onto a malware drop (or any other malicious) page. Alternatively, the attacker might redirect the browser to targetsite.com/deleteprofile?confirm=1.

It is worth mentioning, that stuffing unsanitized user-defined input into an HTTP header might lead to header injection which is pretty bad.

Prevention: Options include:

  • Don’t do redirects at all (they are seldom necessary).
  • Have a static list of valid locations to redirect to.
  • Whitelist the user-defined parameter, but this can be tricky.

Epilogue

I hope that I have managed to tickle your brain a little bit with this post and to introduce a healthy dose of paranoia and web security vulnerability awareness.

The core takeaway here is that age-old software practices exist for a reason and what applied back in the day for buffer overflows, still apply for pickled strings in Python today. Security helps you write correct(er) programs, which all programmers should aspire to.

Please use this knowledge responsibly, and don’t test pages without permission!

For more information and more specific attacks, have a look at: https://www.owasp.org/index.php/Category:Attack.

Feedback on this post is welcome and appreciated. Future related posts are planned, particularly on the issue of distributed denial-of-service (DDoS) and old-school (not web) IT security vulnerabilities. If you have a specific request on what to write about, please feel free to contact me directly at [email protected].

Cheers.

This post originally appeared on the Toptal Engineering Blog

Security Vulnerability Assessment Process and Policy

Overview:
In order to maintain high security standards, identify potential vulnerabilities and evaluate the effectiveness of various security controls that were implemented within the infrastructure, it is crucial to perform periodic security assessments.

Goal:
This procedure defines the controls and steps that are required for identifying security vulnerabilities and ensuring reasonable level of security for the infrastructure and application levels.

Process:

External Facing:

  1. Perform automated external application level scans on a daily basis for website and application. (e.g. McAfee Secure, Acunetix).
  2. Perform automated external network level scans on a weekly basis (e.g. McAfee Secure)
  3. Perform in-house, half automated scans with a vulnerability assessment tool (e.g. Qualys)
  4. Execute a dedicated application level and network penetration test by a professional third party.
    This should be executed twice a year or on every major application release.

Internal:

  1. Discovery: run NMAP scan on all VLANs to identify all the devices and create an asset inventory that outlines devices and services. [weekly / monthly]
  2. Network and Infra vulnerabilities: Run a weekly scan with NESSUS or similar tool to identify infrastructure gap and non hardened devices.
  3. Purchase and run vulnerability scanner (such as Qualys or NetIQ) – every week.
  4. Patch Management:
    • Install Microsoft WSUS server to maintain security patches for Windows infrastructure.
    • Install Linux YUM server to maintain security patches for RedHat infrastructure.
    • Generate reports on weekly basis to find vulnerable systems.
  5. Penetration test: run an annual internal pen-test to identify internal gaps with orientation to threats from within the organization.

Implement a Production Change Management policy that includes a hardening and implementation clearance process for new devices (e.g. addition of new network device, operating system, web server, DB server, etc).