tiscaf http server, manual

This is a tiscaf http server manual. I hope the manual you are reading is both short and complete.

See HomeServer.scala also - it's a short demo.

HServer

Trait HServer provides:

to implement

protected def apps : Seq[HApp] - you must supply a list of web applications (see HApp below). Take in mind, an order of applications in the list is important for dispatching (is explained below also).

protected def ports : Seq[Int] - ports to listen to. Dedicated TCP connection acceptor will be started in own thread for each port.

protected def talkPoolSize : Int, protected def talkQueueSize : Int - parameters of executor in which all requests handlers activity takes place. I don't see any reason to make talkPoolSize (threads count) noticeably greater rather CPU (cores) count. For Core 2 Duo, which I use, 4 is a "good" number - it keep all other parts of operating system absolutely responsive. As for talkQueueSize, well, it must be sufficient to serve your clients. You may start from Int.MaxValue and reduce it at case of DoS attacks and/or RAM deficit.

protected def selectorPoolSize : Int - reading and writing from/to socket channels take place in another execution pool. Again, there are no performance reasons to rise this thread count above CPU/core count, but at case you have clients with poor reading of the server response you may want to increase this parameter. You will not get better performance, but will be able to deal with such lazy-reading clients.

to override

protected def stopPort : Int = 8911 - at server start listener to this port is also started, waiting for "stop" command to shutdown. Hasn't any sense if you override startStopListener method (see below).

protected def readBufSize : Int = 512 - nio buffers size for requests reading. Increase it at case of frequent multi-MB uploads (will eat more RAM).

protected def writeBufSize : Int = 4096 - nio buffers size for responses writing. Increase it at case of frequent multi-MB downloads (will eat more RAM). Say, with 64K write buffers I have got above 400 MB/sec transfer rate (ApacheBench and the server were running on the same PC with C2D 2.4GHz).

protected def tcpNoDelay : Boolean = false - TCP socket's parameter. It is false by default. The only reason to set to true is the case of benchmarking of a single (or few) clients (say, you can get ~25 requests per second only for single-thread client with not-persistent connection when false is set). Set the parameter to false in production to make TCP/IP stack more happy.

protected def connectionTimeoutSeconds : Int = 30 - it has two purposes:

protected def interruptTimeoutMillis : Int = 1000 - after the server has got 'stop' command, this period will be given to handlers to terminate their work before interrupting them.

protected def onError(e : Throwable) : Unit - you can delegate error handling to your favourite logging system. Probably, some kind of filtering may be useful - say, when client interrupts a connection, you may get 'broken pipe' or something such.

protected def startStopListener : Unit - the method is calling during server starting. By default it starts (in dedicated thread) a primitive port listener which waits for 'stop' sequence from HStop.stop. If you need more elaborated shutting down procedure, you can override the method and write your own 'stopper' and stop-listener.

API

final def start : Unit, final def stop : Unit - self explained.

HApp

Presents something called 'application' - a group of request handlers (HLets) sharing common behaviour.

to implement

def resolve(req : HReqHeaderData) : Option[HLet] - core dispatching method, returning a handler to process the request. To decide which handler to use, you have full request header information presented by HReqHeaderData:

trait HReqHeaderData {
  def reqType : HReqType.Value
  def host    : Option[String]
  def port    : Option[String]
  def uriPath : String
  def uriExt  : Option[String]
  def query   : String
  
  def header(key : String): Option[String]
  def headerKeys : scala.collection.Set[String]
  
  def contentLength : Option[Long]
  def isPersistent : Boolean
  def boundary : Option[String]
}

uriExt is an URI path extension - when you use sessions with URL-rewriting, you have URIs like '.../index.html;sid=bla-bla-bla', where 'sid=bla-bla-bla' is an uriExt.

All (request and response) headers keys are case-insensitive.

HReqType is defined as

object HReqType extends Enumeration {
  val Invalid    = Value("Invalid")
  val Get        = Value("GET")
  val PostData   = Value("POST/application/x-www-form-urlencoded")
  val PostOctets = Value("POST/application/octet-stream")
  val PostMulti  = Value("POST/multipart/form-data")
}

Request parser falls back to HReqType.PostOctets type when content type is another rather application/x-www-form-urlencoded or multipart/form-data, delegating data processing to handler.

As I have already said, HServer.apps list order is important. Instead of plenty of words, let's see how dispatching works:

protected object HResolver {

  private object errApp extends HApp {
    // some code to define the HApp
    val hLet = new let.ErrLet(HStatus.NotFound)
  }
  
  def resolve(apps : Seq[HApp], req : HReqHeaderData) : (HApp, HLet) = {
    
    def doFind(idx : Int) : (HApp,HLet) = 
      if(idx == apps.size) (errApp, errApp.hLet) 
      else apps(idx).resolve(req) match {
        case Some(let) => (apps(idx), let)
        case None      => doFind(idx + 1)
      }
    
    doFind(0)
  }
}

You see, if resolver has not found suitable (HApp, HLet) pair, default 'not found' response will be responded.

You can have last HApp in HServer.apps list with your own 'not found' handler (or, say, return 'Charlie Parker - Summertime.ogg'). Inside each HApp.resolve you will probably have some kind of matching against request parameters, and can end up with FsLet (supplied handler to deal with static content) to access images, css, js and other such resources. Also, you can... Ugh, you see, you can everything wrt dispatching.

to override

Also HApp has few params you can override:

  def tracking : HTracking.Value  = HTracking.NotAllowed
  def sessionTimeoutMinutes : Int = 30
  def keepAlive : Boolean         = false
  def chunked : Boolean           = false
  def buffered : Boolean          = false 
  def gzip : Boolean              = false 

They are self-explained. If you are not using buffered (or gzipped as a case of buffered) or chunked output, you need to set content length manually in request handler.

HTracking id defined as:

object HTracking extends Enumeration { 
  val NotAllowed, UriOnly, CookieOnly, UriAndCookie = Value 
}

HTalk

This is your request handler's gate to the world. Note, many methods return this for chaining. HTalk public interface consists of:

request data

  object req {
    // common
    def method : HReqType.Value
    def host : Option[String]
    def port : Option[String]
    def uriPath : String
    def uriExt : Option[String]
    def query : String
    def remoteIp : String 
  
    // header
    def header(key : String): Option[String]
    def headerKeys : scala.collection.Set[String]
  
    // parameters
    def paramKeys : Seq[String]
    def params(key : String) : Seq[String]
    def param(key : String) : Option[String]
    
    // POST/application/octet-stream case
    def octets : Option[Array[Byte]]
  }

Self-explained.

setting response header

  def setStatus(code : HStatus.Value) : HTalk
  def setStatus(code : HStatus.Value, msg : String) : HTalk
  
  def setHeader(name : String, value : String) : HTalk
  def removeHeader(name : String) : Option[String]
  def getHeader(name : String) : Option[String]
  
  def setContentLength(length : Long) : HTalk
  def setCharacterEncoding(charset : String) : HTalk
  def setContentType(cType : String) : HTalk

Self-explained again. Recall, all headers-related keys are case-insensitive.

output, close

  def write(ar : Array[Byte], offset : Int, length : Int) : HTalk
  def write(ar : Array[Byte]) : HTalk
  def close : Unit
  def isClosed : Boolean

It is safe to close a talk multiple times.

At first I have used OutputStream interface, but have rejected it because of semantic difference.

session

  object ses {
    def tracking : HTracking.Value
    def isAllowed : Boolean
    def isValid : Boolean
    def clear : Unit
    def invalidate : Unit
    
    def idKey : String    // "sid"
    def id : String
    def idPhrase : String // it is like ";sid=bla-bla-bla" - for, say, templating
  
    def apply(key : Any) : Option[Any]
    def update(key : Any, value : Any) : Unit
    def keys : Seq[Any]
    def remove(key : Any) : Option[Any]
  }

Note, you use apply and update to get/set session data.

HLet

It is a request handler.

to implement

def act(talk : HTalk) : Unit - request handling. Main your code is here. Just use HTalk.

to override

def before : Seq[HLet] = Nil - you can list other HLets here, whos act methods will be executed as long as HTalk isn't closed. Say, you can use it for access logging, authorisation checking and such. Just look at the server internal implementation fragment:

  private def talk : Unit = {
    // some code
    if(let.before.find( be => { be.act(tk); tk.isClosed }).isEmpty) let.act(tk)
    tk.close // if user didn't
  }

As you can see, there isn't a recursion wrt calling act of before list items. I have decided such recursion will provoke over-complicated design and result in confusion.

Also, you can see, your act method will be called if nobody closed a talk.

def paramsEncoding : String = "UTF-8" - self-explained.

def partsAcceptor(reqInfo : HReqHeaderData) : Option[HPartsAcceptor] = None - is used for handling multipart request, which is executed before act. HPartsAcceptor looks like

abstract class HPartsAcceptor(reqInfo : HReqHeaderData) {
  
  // to implement
  
  def open(desc : HPartDescriptor) : Boolean // new part starts with it...
  def accept(bytes : Array[Byte]) : Boolean  // ... takes bytes (multiple calls!)...
  def close :Unit                            // ... and ends with this ...
  def declineAll : Unit                      // ... or this one apeals to abort all parts
  
  // to override
  
  def headerEncoding : String = "UTF8" 
}

trait HPartDescriptor {
  def header(key : String) : Option[String]
  def headerKeys : scala.collection.Set[String]
  override def toString = (for(k <- headerKeys) yield { k + " -> " + header(k).get }).mkString(", ")
}

As you can imagine, for each part open, accept (few times) and close methods will be called during request parsing. If you want to decline this data stream (say, client doesn't respect max upload size), just return false - the connection will be closed.

Also a request parser may deside input stream is illegal and call declineAll, which you can use to clean up any resources (say, close files).

API

There are few helper methods you can use during talking. Some of them are self-explained:

  protected def error(status : HStatus.Value, msg : String, tk : HTalk) = new let.ErrLet(status, msg) act(tk)
  protected def error(status : HStatus.Value, tk : HTalk)               = new let.ErrLet(status) act(tk)
  protected def e404(tk : HTalk)                                        = error(HStatus.NotFound, tk)

Take in mind, talk will be closed after these methods calling.

Some of helpers need few words:

protected def redirect(to : String, tk : HTalk) = new let.RedirectLet(to) act(tk) - uses helper handler - RedirectLet - which act method looks as

  def act(tk : HTalk) {
    tk.setContentLength(0)
      .setContentType("text/html")
      .setHeader("Location", toUrl)
      .setStatus(HStatus.MovedPermanently)
      .close
  }

You see, just Location header is used.

protected def delegate(to : HLet, tk : HTalk) : Unit - delegates acting to another HLet:

  protected def delegate(to : HLet, tk : HTalk) : Unit =
    if(to.before.find(be => { be.act(tk); tk.isClosed }).isEmpty) to.act(tk)

Take in mind, the delegation respects before list of the target HLet.

protected def sessRedirect(to : String, tk : HTalk) : Unit - the same as redirect, but inserts URI path extension into the target URI:

  protected def sessRedirect(to : String, tk : HTalk) : Unit =  {
    val parts = to.split("\\?", 2)
    val url = parts(0) + ";" + tk.ses.idKey + "=" + tk.ses.id + {
      if(parts.size == 2) "?" + parts(1)
      else ""
    }
    new let.RedirectLet(url) act(tk)
  }

FsLet

There are few HLets located in the let package. You have already seen ErrLet and RedirectLet. There is another useful handler which handles requests to static (file system based) content. It is (surprise?) FsLet handler:

private object FsLet {
  val stdIndexes = List("index.html", "index.htm")
}

trait FsLet extends HLet {
  
  //----------------- to implement -------------------------
  
  protected def dirRoot : String // will be mounted to uriRoot

  //----------------- to override -------------------------
  
  protected def uriRoot : String         = "" // say, "myKit/theDir"
  protected def indexes : Seq[String]    = stdIndexes 
  protected def allowLs : Boolean        = false
  protected def bufSize : Int            = 4096
  protected def plainAsDefault : Boolean = false
  
  // internals follow...
}

To implement:

protected def dirRoot : String - it is your file system path to static content root directory, say, '/home/thelonious/my-site/img'.

To override:

protected def uriRoot : String = "" - this is an URI path prefix the dirRoot will be mounted to. Say, "img", or "", or "a/b/c" (warning for ms windows users: please, follow standards and do not use back slash here).

protected def indexes : Seq[String] = stdIndexes - self explained (see FsLet.stdIndexes above). Probably, you will want Nil here.

protected def allowLs : Boolean = false - at true case standard directory listing html will be returned for directory, as you probably have noticed playing with HomeServer demo.

protected def bufSize : Int = 4096 - files are read to Array[Byte] with this size, then the buffer is used in HTalk.write call.

protected def plainAsDefault : Boolean = false - few MIME types are listed inside HData.scala. At case a type is unknown, this code takes place:

  if(plainAsDeault) tk.setContentType("text/plain")
  else tk.setContentType("application/octet-stream")