Rate Limiting

About

State of the art rate-limiting in Java. Implemented algorithms:

• Token bucket algorithm
• Fixed window algorithm
• Sliding window log algorithm

Highly customizable and extensible implementation with assumptions about the environment used - it can be easily extended to be used with any key-value storage backend such as:

• Hazelcast
• Redis
• ...

See Hazelcast or JCache example storage implementation

Features

• Multiple policies per user
• Blazing speed
• Multiple algorithms per user
• Support for distributed environments
• Pluggable storage backend system
• Generic storage key types

Usage

To perform Rate Limiting implement RateLimiter interface or use existing RateLimiterImpl. You can implement use your key-value database by implementing StorageBackend interface or use the existing HazelcastStorage implementation.

Examples

Simple example

StorageBackend<String> storageBackend = new InMemoryStorageBackend<>(); // in memory impl.
EntryStorage entryStorage = new DistributedEntryStorage(storageBackend); // async mode
RateLimiter rateLimiter = RateLimiting.withStorage(entryStorage);

if (rateLimiter.conformsRateLimits("userIdentifier")) {
    System.out.println("User has no policies so this will be printed!");
} else {
    System.out.println("Too many requests!");
}

Advanced rate limit filter example (javax)

Full source

public abstract class RateLimitFilter implements ContainerRequestFilter {

    @Override
    public void filter(ContainerRequestContext req) throws IOException {
        try {
            Optional<String> identifier = getIdentifier(req);
            if (!identifier.isPresent()) {
                return;
            }

            ConsumptionEntry consumptionEntry = getRateLimiter().conformRateLimitsWithConsumption(identifier.get());
            long retryAfter = TimeUnit.NANOSECONDS.toMillis(consumptionEntry.getNanosUntilConsumption());

            // Inject headers
            response.addHeader(RATE_LIMIT_REMAINING_HEADER,
                    String.valueOf(consumptionEntry.getRemainingTokens()));
            response.addHeader(RETRY_AFTER_HEADER, String.valueOf(retryAfter));

            if (!consumptionEntry.doesConform()) {
                req.abortWith(createRateLimitResponse(consumptionEntry));
            }
        } catch (RateLimiterException ex) {
        }
    }

}

If you need custom serialization combined with your custom storage-backend extend base classes e.g. SimpleRefillPolicy, AbstractRecord and AbstractEntry and implement required serialization methods.

Configuration

Env variables

• ratelimit.map.users.limits: Hazelcast IMap name (default ratelimit.map.users.limits)
• distributedStorageBackendTimeout: Timeout for rate limiter pass-through mode in ms (default 500ms). You should decrease this in production to avoid long latencies in case of StorageBackend failures.

Scheduling

It turns out rate limiting algorithms are very appropriate for scheduling.

Scheduling example

EntryBuilder builder = RateLimiting.schedulerBuilder().withAlgorithm(RateLimitAlgorithm.TOKEN_BUCKET);
RefillPolicy policy = SimpleRefillPolicy.perSecond(2);
RateLimitEntry record = builder.withRefillPolicy(policy).build();

long start = System.currentTimeMillis();
while (record.tryConsume(1)) {
	double secondsPassed = (System.currentTimeMillis() - start) / 1000.0;
	System.out.println(secondsPassed); // or someVeryExpensiveTask();
}

Output:

0.502
1.004
1.504
2.006
2.508
3.011
...