Streaming File Upload with Erlang and Mochiweb Multipart Post

Posted:	2007-12-31 19:13
Tags:	Web, Erlang

Here's one way to set up a simple Mochiweb server to handle a file upload using Erlang.

On Debian you'll need to install the following:

sudo apt-get install build-essential libncurses-dev libssl-dev m4 subversion

(I've also got autoconf, automake and libgd-dev so you might need those too).

First install the latest version of Erlang:

wget http://erlang.org/download/otp_src_R12B-0.tar.gz
tar zxfv otp_src_R12B-0.tar.gz
cd otp_src_R12B-0

I already have Erlang installed as part of the Debian distribution so I install this version to my home directory so it doesn't conflict:

./configure --prefix=/home/james

You'll get an error similar to this but we won't be using these features so we can egnore the warning:

*********************************************************************
**********************  APPLICATIONS DISABLED  **********************
*********************************************************************

jinterface     : No Java compiler found
odbc           : No odbc library found
percept        : libgd not working

*********************************************************************

Now make and install Erlang:

make
make install

This takes about 3 minutes to complete on a fairly modern dual core server.

You'll probably want the html and man pages too:

cd /home/james/lib/erlang
wget http://erlang.org/download/otp_doc_man_R12B-0.tar.gz
wget http://erlang.org/download/otp_doc_html_R12B-0.tar.gz
gunzip -c otp_doc_man_R12B-0.tar.gz | tar xf -
gunzip -c otp_doc_html_R12B-0.tar.gz | tar xf -
rm otp_doc_man_R12B-0.tar.gz otp_doc_html_R12B-0.tar.gz

Now that Erlang is set up get the latest trunk of Mochiweb:

svn checkout http://mochiweb.googlecode.com/svn/trunk/ mochiweb

You'll need to compile it:

export PATH=/home/james/bin:/home/james/lib:$PATH
cd mochiweb
make

Now setup a project called file_upload like this:

cd scripts
chmod a+x new_mochiweb.erl
./new_mochiweb.erl file_upload /home/james/

Compile it and run it:

make
./start-dev.sh

The server will be accessible on port 8080. You can stop it by pressing Ctrl+c and then typing the letter a followed by return. Try visiting http://yourserver.com:8000/index.html and you should see the message MochiWeb running..

You now need to modify the src/file_upload_web.erl file for our file upload application. Underneath the -export() line near the top add this:

-record(state, {filename, file}).

callback(Next, State) ->
    case Next of
        {headers, Headers} ->
            % Find out if it is a file
            [ContentDisposition|_] = Headers,
            NewState = case ContentDisposition of
                {"content-disposition", {"form-data",[{"name",_},
                    {"filename",Filename}]}} ->
                    #state{filename="/tmp/"++Filename};
                _ ->
                    State
            end,
            fun(N) -> callback(N, NewState) end;
        {body, Data} ->
            if  State#state.filename =/= undefined ->
                if State#state.file =/= undefined ->
                    file:write(State#state.file, Data),
                    NewState = State;
                true ->
                    case file:open(State#state.filename, [raw,write]) of
                        {ok, File} ->
                            file:write(File, Data),
                            NewState = State#state{file=File};
                        {error, Error} ->
                            io:format(
                                "Couldn't open ~p for writing, error: ~p~n",
                                [State#state.filename, Error]),
                            NewState=State,
                            exit(could_not_open_file_for_writing)
                    end
                end;
            true ->
                NewState = State
            end,
            fun(N) -> callback(N, NewState) end;
         body_end ->
            if State#state.file =/= undefined ->
                file:close(State#state.file);
            true ->
                ok
            end,
            fun(N) -> callback(N, #state{}) end;
         _ ->
            fun(N) -> callback(N, State) end
    end.

Then replace the loop function with this:

loop(Req, _) ->
    case Req:get(method) of
        Method when Method =:= 'GET'; Method =:= 'HEAD' ->
         Req:ok({"text/html", [], <<"<html><body><h1>File Upload</h1>
<form enctype=\"multipart/form-data\" action=\"/\" method=\"post\">
<label for=\"file\">File:</label>
<input type=\"file\" name=\"file\" id=\"file\"/>
<input type=\"submit\" name=\"upload\" value=\"Upload\" />
</form>
</body></html>">>});
        'POST' ->
                Callback = fun(N) -> callback(N, #state{}) end,
        mochiweb_multipart:parse_multipart_request(Req, Callback),
        Req:ok({"text/html", [], <<"<html><body><h1>File Upload</h1>
<p>Uploaded successfully.</p>
</body></html>">>});
        _ ->
            Req:respond({501, [], []})
    end.

You'll need to build the new code by running make again in the file_upload directory. The server should automatically restart with the message Reloading file_upload_web ... ok. - if it doesn't you'll need to manually stop and start the server.

If you run the application again you should now have a simple file upload application which saves files to /tmp.

(view source)