Asaf Shakarchi

Software development, hmm... software development :-)

Hello World Using Ratpack.

About Ratpack

Ratpack is a set of Java libraries that facilitate fast, efficient, evolvable and well tested HTTP applications built on Netty event driven networking engine,

After crawling into its source code, I find it simple and easy to use and it feels right, why? because:

  1. It’s super simple, its core has very limited number of dependencies
  2. Has out of the box extensions such as Guice, Jackson, etc, but nothing dictates you using them thanks to Guice modular architecture.
  3. Compatible with Java8 lambda syntax
  4. Testing is very easy.

Code can be written in Groovy or Java, and since java8 lambda syntax is so pretty, I find it suitable for the Ratpack handlers architecture quit well.

So here’s a simple hello world using Ratpack and Java8,

The build file (Gradle)

While used to Maven, It feels like Ratpack folks like Gradle, so I said why not use it?

Here’s a simple build.gradle file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
buildscript {
  repositories {
    jcenter()
  }
  dependencies {
    classpath "io.ratpack:ratpack-gradle:0.9.13"
  }
}

apply plugin: "io.ratpack.ratpack-java"
apply plugin: "idea"

dependencies {
  //Ratpack uses slf4j for logging
  runtime "org.slf4j:slf4j-simple:1.7.5"
}

//we need this since 0.9.13 is based on snapshot version of Netty
repositories {
  jcenter()
    maven {
      url "http://clinker.netty.io/nexus/content/repositories/snapshots"
    }
}

Create basic directories

1
2
mkdir -p src/main/java
mkdir -p src/ratpack

Run in the command like gradle idea to create Intellij project:

1
gradle idea

One of the things I mostly like about Ratpack is that it has no real notion of a “container”, you can simply start it a plain main method just like that:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import ratpack.server.RatpackServer;
import ratpack.server.ServerConfig;

/**
 */
public class Server {
    public static void main(String[] args) throws Exception {
        RatpackServer server = RatpackServer.of(b -> b
                        .serverConfig(ServerConfig.embedded()) //default config
                        .registryOf(r -> r.add(String.class, "world"))  // registry of supporting objects - optional
                        .handlers(chain -> chain                    // request handlers - required
                                        .get("hello", ctx -> ctx.render(ctx.get(String.class) + " !"))
                        )
        );

        server.start();
    }
}

Server will start with on some available port and will print: Ratpack started for http://localhost:65478

now hit in the browser http://localhost:65478/hello ! :)

Choosing Java Backend Framework in 2015.

So it’s year 2015 and I’m taking a role in a new cloud service, helping to choose the right backend technology,

This is the summary of our requirements:

  1. Can quickly come up with some POC.
  2. HTTP driven.
  3. Native support for Async.
  4. Modular.
  5. Avoid frameworks that dictates specific code structure.

Here are few options we tried but decided to pivot and seek for alternatives,

Akka

Akka seems very promising, it feels lightweight, highly concurrent and distributed, Akka is based on actors pattern, an old pattern invented by Carl Hewitt and friends, promoted lately by Erlang,

We tried to create a POC with Akka, in conjuction with their new Akka Streams (which seems to replace Spray), both felt like a complete solution, but we decided to pivot out of it, I think I can summarize it in one sentence.

Although Akka seems to have support for Java, it’s just not a java project, Akka is written in Scala, and exposes Java APIs via bindings layers, I felt like its community is Scala oriented debugging code of Scala is something tough for people that plans to stick in the Java domain.

Vert.x

Vert.x is similar to NodeJs, written in Java, It is Netty based and works in a non blocking mode,

Vertx 3 seems to be very promising we found some advantages of it,

  1. We quickly could create a web app using Apex.
  2. Almost no configuration and deployment code is required to see real results.
  3. Completely written in Java (which was a big releaf after Akka)
  4. Scale out by levaraging an internal even bus.

But we decided to pivot again, but why?! because Vert.x makes certain irreversible decisions about how applications should be written, and working with the event bus was a nightmare, we tried to use Vertx Service Factory which eliminates most of the boilerplate code required to perform actions over the event bus, but it restrictions of passing only JsonObject and JsonArray frustrated us, We found ourself spending too much time on convertions between Vertx’s JsonObjects and application / core services models.

What now?

So what now? we’r still seeking for a better solution that hopefully will satisfy our needs.

Ember Forms

Smart, Intuitive forms for Ember.js styled with Bootstrap, Multi layouts and Validation support.

As my favorite web framework is emberjs, I lately needed forms support for one of my projects with the following set of features in mind:

  • Styled for Bootstrap but easy to stil differently.
  • Different layouts such inline and horizontal
  • Validations where validation rules are defined in model/controller and form supports for:
    • Change color & icons when validation rule fails.
    • Show error message per form input.
    • Submit button with async support that changes its styles & get disabled during async call.

So here is the result: https://github.com/indexiatech/ember-forms

Demo & docs: http://indexiatech.github.io/ember-forms

Bootstrap for Ember Is Born

Bootstrap is an amazing set of pure CSS3 / HTML 5 components, while EmberJs amazes in its beauty of making web applications right.

You can just hook both projects together by using Bootstrap as is in Handlebars templates, But soon enough, you will find yourself asking questions sucha s:

  • How do I interact between Bootstrap events and Ember (such as Controllers)?
  • How to pass Bootstrap components data from Ember (such as Controllers)

I’ve seen few options about the integration between both, but:

  • They are not easy and fun to use, they make the ease of Bootstrap too complex.
  • They are not well documented.
  • Development is too slow, I haven’t seen anything new lately.

So! I’m excited about a project I’ve been working lately with some guys to integrate Bootstrap & EmberJs together in a way that doesnt make Bootstrap usage more complex than using it as-is but providing great integration with EmberJs components.

Links!

Nodejs-model Is Born

I’ve been struggling lately to find an appropriate way to handle models correctly, In one hand, these noSQL DBs such CouchDB have great flexibility and I didn’t want to ruin that with a blowing up ORM framework, in the other hand once the application grows a little bit more than a contact list app, then things become messy.

While writing a larger app with Node, I find myself struggling with finding solutions to problems such: (there are few more but these are the main)

  • Validating a document (model) properties (application level) before storing it in the DB.
  • A minimal schema where each attribute is defined per model to ensure no allowed attributes leaks into the model.
  • Automatically convert a property value when a value is set.
  • Filter critical, sensitive attributes when transfering the model on the wire to external clients.
  • Updating an existing domain via an accessibility-aware method (you may want users to update their profile but not letting them directly change their password attribute)

So I couldn’t really find a good module to achieve all of that without sticking a giant into my lightweight app, so I built one, it’s named nodejs-model, available here: https://github.com/asaf/nodejs-model

For now, read the docs on the github repo for more info, i will post series of posts about nodejs-model in the near future.

Nodejs Deployment: Building and Configuring on Amazon Linux AMI

Logging in and updating system to latest

SSH your shiny new VM,

Now lets update the system to the latest:

1
sudo yum update

Install OS dependencies

We’r going to build Node.js from sources, some dependencies (such as gcc) are required:

1
sudo yum install gcc-c++ make openssl-devel git

Cloning n Building Node.js

1
2
3
4
5
6
7
cd
git clone git://github.com/joyent/node.git
cd node
git checkout v0.10.13 #check for other stable tags by executing 'git tag'
./configure --prefix=/usr/local/node
make
sudo make install

Go grab a cup of coffee… :–) …

Configuration

1
2
3
sudo useradd _yourappuser_
passwd _yourappuser_
sudo su - _yourappuser_

Put your app

Now put your app in ~, for instance:

1
2
3
4
cd
pwd
#/home/_yourappuser_
git clone _https://myhost.com/myapp myapp_

Init.d

We would like to have nodejs to start automatically as a service, to do so, lets create an init.d file Note: you have to change the properties in the file such as yourappuser, myapp to your app folder and server.js to your node app file.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
cat << 'EOF' > /etc/init.d/nodejs

#!/bin/sh

#
# chkconfig: 35 99 99
# description: Node.js /home/yourappuser/myapp/app.js
#

. /etc/rc.d/init.d/functions

USER="_yourappuser_"

NODE_ENV="production"
DAEMON="/usr/local/node/bin/node"
ROOT_DIR="/home/yourappuser/myapp"

SERVER="$ROOT_DIR/server.js"
LOG_FILE="$ROOT_DIR/app.js.log"

LOCK_FILE="/var/lock/subsys/node-server"

do_start()
{
        if [ ! -f "$LOCK_FILE" ] ; then
                echo -n $"Starting $SERVER: "
                runuser -l "$USER" -c "NODE_ENV=$NODE_ENV $DAEMON $SERVER >> $LOG_FILE &" && echo_success || echo_failure
                RETVAL=$?
                echo
                [ $RETVAL -eq 0 ] && touch $LOCK_FILE
        else
                echo "$SERVER is locked."
                RETVAL=1
        fi
}
do_stop()
{
        echo -n $"Stopping $SERVER: "
        pid=`ps -aefw | grep "$DAEMON $SERVER" | grep -v " grep " | awk '{print $2}'`
        kill -9 $pid > /dev/null 2>&1 && echo_success || echo_failure
        RETVAL=$?
        echo
        [ $RETVAL -eq 0 ] && rm -f $LOCK_FILE
}

case "$1" in
        start)
                do_start
                ;;
        stop)
                do_stop
                ;;
        restart)
                do_stop
                do_start
                ;;
        *)
                echo "Usage: $0 {start|stop|restart}"
                RETVAL=1
esac

exit $RETVAL

EOF

Add execution permission to the nodejs init script

1
sudo chmod +x /etc/init.d/nodejs

Pre Routing to port 80

Linux does not allow non super users to listen to ports < 1024, assuming your application listen to port 8080, You would probably like to pre route any traffic arriving from port 80 to your node app that listens to port 8080

You can do this by the pre routing nat capability of Iptables

1
2
3
4
5
chkconfig iptables on
service iptables start
iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8080
iptables -t nat -A PREROUTING -p tcp --dport 443 -j REDIRECT --to-port 8443 #if you want SSL too
service iptables save

Configuring node-http-proxy

It is common to install http proxies such as nginx on front of nodejs, This architecture has many advantages such as raising security level, listening natively to port 80, load balancing, multiple node apps support via url rewrite, etc…

I personally think that the best approach, which is very native to node apps is to use https://github.com/nodejitsu/node-http-proxy,

Which have several advantages:

  1. Reverse proxies incoming http.ServerRequest streams, WebSockets, HTTPS
  2. Minimal request overhead and latency
  3. Battled-hardened through production usage
  4. Very native for nodejs apps

TODO: Will post more details in the future but you can simply visit “https://github.com/nodejitsu/node-http-proxy” site.

https://github.com/pkrumins/nodejs-proxy

Installing CouchDB Instance on Amazon AMI Using Build-couchdb

This post explains how to install CouchDB on Amazon AMI using build-couchdb/

If you want to install couch from sources without build-couchdb tool, read: Installing CouchDB instance on Amazon AMI is a working link

build-couchdb does the following things:

  1. Install general dependencies via yum (such as gcc, ruby, automake, rubygems, etc.)
  2. Clone the ‘build-couchdb’ project, initalize and update its sub modules (will clone CouchDB itself, curl, gnulib, etc), These are all the dependencies that CouchDB needs in order to be compiled.
  3. Compiles all dependencies
  4. Compiles CouchDB linked with the compiled dependencies.

Okay, go grab coffee and lets start!

Launch Instance

Login into EC2, and launch a new instanc and choose Amazon Linux AMI, You can of course choose other Linux flavours (you are not going to use Windows right?),

But I really suggest going with Amazon AMI, it is EBS backed, ships with AWS tools installed and has a RHEL style package management using yum.

Once your instance is launched, SSH it, i’m sure you can do.

Now lets updated the system to latest:

1
sudo yum update

Installation

Installing OS dependencies

1
2
sudo yum install gcc gcc-c++ libtool curl-devel python27 help2man texinfo ruby-rdoc zlib-devel openssl-devel make automake rubygems perl git-core
sudo gem install rake --no-ri --no-rdoc

If you don’t want to get into troubles where build-couch tries to patch autoconf-2.69 because of old python version (< 2.6) then link to python 2.7, at least during the time couch is being built.

1
2
sudo mv /usr/bin/python /usr/bin/python_org
sudo ln -s /usr/bin/python2.7 /usr/bin/python

Clone build-couchdb

git clone git://github.com/iriscouch/build-couchdb cd build-couchdb git submodule init git submodule update

Optional: Upgrade SpiderMonkey

build-couch uses not an official Mozilla SpiderMonkey version, but considered to be very stable, if you would like to replace it with other versions then do the following, You can safely skip this, proceed only if you have a good reason to do so

1
2
3
4
5
6
7
8
sudo yum install mercurial
cd
hg clone --verbose http://hg.mozilla.org/releases/mozilla-release
cd mozilla-release
hg checkout FIREFOX_XX_0_RELEASE --clean #replace XX with the tag you wish
rm -rf ~/build-couchdb/dependencies/spidermonkey/js
rm -rf ~/build-couchdb/dependencies/spidermonkey/mfbt
cp -r js mfbt ~/build-couchdb/dependencies/spidermonkey

Building CouchDB and its dependencies

Now lets build Couch, you can replace 1.3.1 with any CouchDB release you wish to build, Or change the installation location.

1
2
sudo rake git="git://git.apache.org/couchdb.git tags/1.3.1" install=/usr/local/couchdb-1.3.1
sudo ln -s /usr/local/couchdb-1.3.1 /usr/local/couchdb

Now go eat something, drink (another?) coffee, take a shower (you will have time for all of them, I promise.)

You should expect the following output:

Plugins done

Now lets see what we’ve built:

1
2
3
4
5
6
7
8
9
cd /usr/local/couchdb-1.3.1
ls bin

> aclocal       autoheader2.13  autom4te2.69    autoreconf2.69  autoupdate2.59  ct_run       erl       genctd       ifnames2.62  pkgdata
> aclocal-1.11  autoheader2.59  automake        autoscan2.13    autoupdate2.62  curl         erlc      genrb        ifnames2.69  run_erl
> autoconf2.13  autoheader2.62  automake-1.11   autoscan2.59    autoupdate2.69  curl-config  escript   icu-config   js-config    run_test
> autoconf2.59  autoheader2.69  autoreconf2.13  autoscan2.62    couch-config    derb         genbrk    icuinfo      libtool      to_erl
> autoconf2.62  autom4te2.59    autoreconf2.59  autoscan2.69    couchdb         dialyzer     gencfu    ifnames2.13  libtoolize   typer
> autoconf2.69  autom4te2.62    autoreconf2.62  autoupdate2.13  couchjs         epmd         gencnval  ifnames2.59  makeconv     uconv

Lets execute Couch to ensure it can be executed.

1
2
3
4
5
sudo ./bin/couchdb

> Apache CouchDB 1.3.1 (LogLevel=info) is starting.
> Apache CouchDB has started. Time to relax.
> [info] [<0.32.0>] Apache CouchDB has started on http://127.0.0.1:5984/

You can open another terminal to ensure Couch replies properly:

1
2
3
curl http://127.0.0.1:5984

> {"couchdb":"Welcome","uuid":"9db29433ea76ccfaa81726f6ace742f4","version":"1.3.1","vendor":{"name":"The Apache Software Foundation","version":"1.3.1"}}

Configuration

Replace Python link

To avoid breaking yum and other things, it is adviced to return the python binary link back to its original version

1
2
sudo rm /usr/bin/python
sudo mv /usr/bin/python_org /usr/bin/python

Run Couch on isolated user and correct files ownership

1
2
3
4
5
# add couchdb user
sudo adduser --system --home /usr/local/couchdb/var/lib/couchdb -M --shell /bin/bash --comment "CouchDB" couchdb

# change file ownership
sudo chown -R couchdb:couchdb /usr/local/couchdb/etc/couchdb /usr/local/couchdb/var/lib/couchdb /usr/local/couchdb/var/log/couchdb /usr/local/couchdb/var/run/couchdb

couch ini file

Lets create couch ini file

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
cat << 'EOF' > /usr/local/couchdb/etc/couchdb/local.ini
[couchdb]
#read more here: http://guide.couchdb.org/draft/performance.html
delayed_commits = false

[httpd]
port = 80
bind_address = 0.0.0.0
socket_options = [{recbuf, 262144}, {sndbuf, 262144}, {nodelay, true}]
WWW-Authenticate = Basic realm="administrator"
;WWW-Authenticate = bummer

[couch_httpd_auth]
require_valid_user = true

[log]
level = error

[admins]
EOF

Password protection

1
2
export PASS=`</dev/urandom tr -dc '12345!@#$%qwertQWERTasdfgASDFGzxcvbZXCVB' | head -c20; echo ""`
echo "admin = ${ADMIN_PASSWORD}" >> /usr/local/etc/couchdb/local.ini

Couch is smart enough to hash the passwords.

Logs Rotation

1
2
3
4
5
6
7
8
if [ ! -e "/etc/logrotate.d/couchdb" ]
then
  # add couch.log to logrotate
  sudo ln -s /usr/local/couchdb/etc/logrotate.d/couchdb /etc/logrotate.d/
  # change to daily rotation
  sudo sed -e s/weekly/daily/g -i /usr/local/couchdb/etc/logrotate.d/couchdb
  #logrotate -v -f /etc/logrotate.d/couchdb 
fi

Run CouchDB on startup

1
2
3
4
5
6
7
8
9
10
# run couchdb on startup
sudo ln -s /usr/local/couchdb/etc/rc.d/couchdb /etc/init.d/couchdb
sudo chkconfig --add couchdb
sudo chkconfig --level 345 couchdb on

sudo service couchdb status
#Apache CouchDB is not running.

sudo service couchdb start
#Apache CouchDB is running as process 5047, time to relax.

Now it’s really time to relax, enjoy :–)