Backend development in Kotlin

backend, caching, database, http, kotlin

Introduction

One of the few issues I find myself facing with being a Kotlin developer and advocate is that it is mainly used for Android development. When you search for anything regarding it - the results are almost always in the context of Android. Search for ‘Kotlin coroutines stack overflow’ and the first result is regarding its usage in Android.

To promote more backend development in Kotlin - I want to share some libraries and frameworks that I typically employ for building APIs. The three pillars I will cover are caching, interacting with a SQL Database, and the HTTP server itself.

HTTP Server

Before any requests from users can be served - we have to get the requests into our system through an entry point. An HTTP API server is a common way of getting a point of ingress for customer requests.

To setup HTTP servers in Kotlin - I tend to use either Javalin or Vert.x . Both Javalin and Vert.x give developer friendly APIs to build REST services. Javalin is based on Jetty, while Vert.x is based on Netty. Javalin employs a typical blocking style request-response pattern [with support for futures] where as Vert.x opts more for a non-blocking node.js style pattern [with support for multi core processing].

A hello world HTTP server that responds to http://localhost:7000/ in Javalin looks something like this

1
2
3
4
5
fun main() {
    val app = Javalin.create()
    app.get("/") { it.result("Hello World") }
    app.start()
}

Hello world server [that returns on any request received] in Vert.x looks something like this

1
2
3
4
5
6
fun main() {
    val vertx = Vertx.vertx()
    vertx.createHttpServer()
        .requestHandler { it.response().end("Hello World") }
        .listen(9000)
}

If starting out - I would start with Javalin because of the familiar blocking style code and keep thing simple. Vert.x, due to its non-blocking nature, may provide better performance and concurrency but has a learning curve and will make hiring, and on-boarding more junior developers to your code-base more difficult.

For most tech shops - Javalin with Jetty should be more than sufficient performance wise.

Something to note is that Javalin is more geared towards ‘just’ the HTTP server part i.e. routing, middleware, websocket handlers etc but Vert.x is a toolkit of libraries that span from networking to data access and observability. Both have great features around validation and request serialisation to POJOs - it really does come down to whether you are ok having a non-blocking style codebase or you want to stick to the traditional blocking style code

Bottom line: Start with Javalin and move to Vert.x if need be.

Caching

The two most difficult things in computer science are naming and cache invalidation.

Caching can help dramatically speed up your application’s performance since the hot data is residing in memory and can be returned quickly. The more of your working data set that can fit in memory the better position you will be in.

There are two libraries that I recommend when adding caching logic to your application.

Caffeine for in app-memory caching and Lettuce for external Redis caching.

A caffeine cache that holds int -> int mappings and expires entries 10 minutes after access and maximum size of 10,000 can be configured as follows. The cache object is thread safe. It is a great way to hold static data [such as session id to user mapping] in memory while still keeping a bound on the number of items.

1
2
3
4
5
6
7
val cache = Caffeine.newBuilder()
    .expireAfterAccess(10, TimeUnit.MINUTES)
    .maximumSize(10_000)
    .build<Int, Int>()

cache.put(1,3)
val data = cache.get(1) { getFromAPI(it) } // getFromAPI is executed if key is not found - it is cached and returned

Redis is a remote dictionary and data structures server. It is probably a tool that is found in most tech stacks. To interact with Redis from your Kotlin backend - I recommend using Lettuce as the library. Lettuce provides sync, async, and reactive style APIs to communicate with redis. Lettuce also supports pub sub notification listeners for redis.

1
2
3
4
5
6
7
val client = RedisClient.create("redis://localhost")
val connection = client.connect()
val redisAPI = connection.sync()

redisAPI.set("foo", "1")
redisAPI.incr("foo")
redisAPI.get("foo")

Database

Data is king in any production system. A battle tested DBMS is critical to any application. To interact with it we need a database client.

My first go to library to interact with a DB is JDBI , which is a convenience layer over JDBC.

1
2
3
4
val jdbi = Jdbi.create("jdbc:postgresql://localhost:5432/postgres")
jdbi.useHandle<Exception> {
   println(it.createQuery("SELECT * from users").mapToMap().first())
}

For production services it is wise to setup a connection pooling library for better connection management - for that HikariCP comes in quite handy

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
val props = Properties()
props.setProperty("dataSourceClassName", "com.impossibl.postgres.jdbc.PGDataSource") // https://github.com/impossibl/pgjdbc-ng
props.setProperty("dataSource.user", "postgres")
props.setProperty("dataSource.password", "password")
props.setProperty("dataSource.databaseName", "postgres")

val config = HikariConfig(props)
config.jdbcUrl = "jdbc:postgresql://localhost:5432/postgres"

val ds = HikariDataSource(config)
val connectionPooledJdbi = Jdbi.create(ds)

connectionPooledJdbi.useHandle<Exception> {
   println(it.createQuery("SELECT * from users").mapToMap().first())
}

You can also use Postgres’ listen/notify feature as follows

1
2
3
4
5
6
7
8
9
10
connectionPooledJdbi.useHandle<Exception> {
  val pgConnection = it.connection.unwrap(PGConnection::class.java)
  val listener = object : PGNotificationListener {
    override fun notification(processId: Int, channelName: String, payload: String) {
      println("received $payload on channel $channelName")
    }
  }
  pgConnection.addNotificationListener("config_changed", listener)
  pgConnection.createStatement().use { statement -> statement.execute("LISTEN config_changed") }
}

The above snippet will run the listener object every time a message gets sent on the config_changed channel. I have triggers set in the database to send a JSON object whenever my config table gets updated.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
CREATE OR REPLACE FUNCTION notify_config_changes()
RETURNS trigger AS $$
BEGIN
  PERFORM pg_notify(
    'config_changed',
    json_build_object(
      'operation', TG_OP,
      'record', row_to_json(NEW)
    )::text
  );

  RETURN NEW;
END;
$$ LANGUAGE plpgsql;

CREATE TRIGGER configs_changed
AFTER INSERT OR UPDATE OR DELETE
ON configs
FOR EACH ROW
EXECUTE PROCEDURE notify_config_changes()

Astute readers might have noticed that there is no new record on a delete operation - to get around that I use a slightly modified version of the above trigger that sends the OLD data on deletes

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
CREATE OR REPLACE FUNCTION notify_config_changes()
RETURNS trigger AS $$
BEGIN
 if (TG_OP = 'DELETE') then
 	PERFORM pg_notify(
    	'config_changed',
	    json_build_object(
    	  'operation', TG_OP,
	      'record', row_to_json(OLD)
    	)::text
	  );
  return OLD;
 end if;
   PERFORM pg_notify(
    'config_changed',
    json_build_object(
      'operation', TG_OP,
      'record', row_to_json(NEW)
    )::text
  );

  RETURN NEW;
END;
$$ LANGUAGE plpgsql;

If you are working in a non-blocking/async style framework such as Vert.x and can not use a blocking library such as JDBI - Vert.x’s reactive PG client is a great alternative. I am using the suspending coroutine bindings to have non-blocking code that still looks and reads like its sequential version

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
val connectOptions = PgConnectOptions()
    .setPort(5432)
    .setHost("localhost")
    .setDatabase("postgres")
    .setUser("postgres")
    .setPassword("password")

// Pool options
val poolOptions = PoolOptions().setMaxSize(8)

// Create the client
val client = PgPool.pool(connectOptions, poolOptions)

val sqlConnection = client.getConnectionAwait()
val rowSet = sqlConnection.query("SELECT * from users").executeAwait()
rowSet.forEach { println(it.getString("username") }
sqlConnection.close()

A good rest server, caching and database client can help you get your MVP to market quickly while still keeping it relatively performant, maintainable, and tech debt free.