SMS Gateway with Orange SMS API, Python, Nginx, mod_wsgi and Postfix

Setting up the hostname

Become root on your server and make sure that /etc/hostname contains the host name without the domain part. The file /etc/mailname should contain the fully-qualified host name with the domain part.

You'll probably need to fix /etc/hosts too. Run hostname --fqdn and see if you get the fully-qualified hostname. If you just get the hostname without the domain please check that your /etc/hosts file has the fully-qualified hostname first in the list.

Wrong:

78.47.146.250   sms sms.3aims.com

Right:

78.47.146.250   sms.3aims.com sms

Setting the Locale

As root:

apt-get install locales

Then if needed:

apt-get install debconf
dpkg-reconfigure locales

Here's what you get:

Package configuration



 ┌──────────────────────────┤ Configuring locales ├──────────────────────────┐
 │                                                                           │
 │ Locale is a framework to switch between multiple languages for users who  │
 │ can select to use their language, country, characters, collation order,   │
 │ etc.                                                                      │
 │                                                                           │
 │ Choose which locales to generate.  The selection will be saved to         │
 │ `/etc/locale.gen', which you can also edit manually (you need to run      │
 │ `locale-gen' afterwards).                                                 │
 │                                                                           │
 │ When `All locales' is selected, /etc/locale.gen will be set as a symlink  │
 │ to /usr/share/i18n/SUPPORTED.                                             │
 │                                                                           │
 │                                  <Ok>                                     │
 │                                                                           │
 └───────────────────────────────────────────────────────────────────────────┘

Select the locale(s) you want:

Package configuration

                     ┌──────┤ Configuring locales ├───────┐
                     │ Locales to be generated:
                     │
                     │    [ ] en_BW ISO-8859-1
                     │    [ ] en_BW.UTF-8 UTF-8
                     │    [ ] en_CA ISO-8859-1
                     │    [ ] en_CA.UTF-8 UTF-8
                     │    [ ] en_DK ISO-8859-1
                     │    [ ] en_DK.ISO-8859-15 ISO-8859-15
                     │    [ ] en_DK.UTF-8 UTF-8
                     │    [ ] en_GB ISO-8859-1
                     │    [ ] en_GB.ISO-8859-15 ISO-8859-15
                     │    [*] en_GB.UTF-8 UTF-8
                     │    [ ] en_HK ISO-8859-1
                     │    [ ] en_HK.UTF-8 UTF-8
                     │
                     │
                     │       <Ok>           <Cancel>
                     │                                    │
                     └────────────────────────────────────┘

I choose to make the new locale the default:

Package configuration

  ┌─────────────────────────┤ Configuring locales ├─────────────────────────┐
  │ Many packages in Debian use locales to display text in the correct      │
  │ language for users. You can change the default locale if you're not a   │
  │ native English speaker. These choices are based on which locales you    │
  │ have chosen to generate.                                                │
  │                                                                         │
  │ Note: This will select the language for your whole system. If you're    │
  │ running a multi-user system where not all of your users speak the       │
  │ language of your choice, then they will run into difficulties and you   │
  │ might want not to set a default locale.                                 │
  │                                                                         │
  │ Default locale for the system environment:                              │
  │                                                                         │
  │                               None                                      │
  │                               en_GB.UTF-8                               │
  │                                                                         │
  │                                                                         │
  │                   <Ok>                       <Cancel>                   │
  │                                                                         │
  └─────────────────────────────────────────────────────────────────────────┘

Setting Up Backports

Read this for background information: http://www.backports.org/dokuwiki/doku.php?id=instructions

The only change is we add the etch backports to the /etc/apt/sources.list:

deb http://www.backports.org/debian etch-backports main

Then run:

wget -O - http://backports.org/debian/archive.key | apt-key add -
apt-get update

Installing Postfix

Install it with this command to explicitly use the backports version:

apt-get -t etch-backports install postfix-mysql

Now you will see a lot of screens:

Package configuration

  ┌────────────────────────┤ Postfix Configuration ├────────────────────────┐
  │                                                                         │
  │ Please select the mail server configuration type that best meets your
  │ needs.
  │                                                                         ▒
  │  No configuration:                                                      ▒
  │   Should be chosen to leave the current configuration unchanged.        ▒
  │  Internet site:                                                         ▒
  │   Mail is sent and received directly using SMTP.                        ▒
  │  Internet with smarthost:                                               ▒
  │   Mail is received directly using SMTP or by running a utility such     ▒
  │   as fetchmail. Outgoing mail is sent using a smarthost.                ▒
  │  Satellite system:                                                      ▒
  │   All mail is sent to another machine, called a 'smarthost', for        ▒
  │ delivery.                                                               ▒
  │  Local only:
  │
  │                                 <Ok>
  │                                                                         │
  └─────────────────────────────────────────────────────────────────────────┘

Click OK then choose Internet Site. You'll see this, enter the domain which will appear after the @ in email addresses:

Package configuration


 ┌─────────────────────────┤ Postfix Configuration ├─────────────────────────┐
 │ The "mail name" is the domain name used to "qualify" mail addresses       │
 │ without a domain name.                                                    │
 │                                                                           │
 │ This name will also be used by other programs. It should be the single,   │
 │ fully qualified domain name (FQDN).                                       │
 │                                                                           │
 │ Thus, if a mail address on the local host is foo@example.org, the         │
 │ correct value for this option would be example.org.                       │
 │                                                                           │
 │ System mail name:                                                         │
 │                                                                           │
 │ sms.3aims.com____________________________________________________________ │
 │                                                                           │
 │                    <Ok>                        <Cancel>                   │
 │                                                                           │
 └───────────────────────────────────────────────────────────────────────────┘

Setting up Logging

Since the scripts we are going to use are executed by either postfix or as a result of an HTTP request from Orange, we won't have any particularly easy way to see what's going on so we'll set up a logging configuration to log data to /var/log/syslog. To do this create a logging configuration which looks like this in /home/sms/sms/logging_basic.ini:

[formatters]
keys: detailed

[handlers]
keys: syslog

[loggers]
keys: root,sms

[logger_root]
level: INFO
handlers: syslog

[logger_sms]
level: DEBUG
handlers: syslog
qualname: sms

[handler_syslog]
class: handlers.SysLogHandler
args: ['/dev/log']
propagate: 0
formatter: detailed

[formatter_detailed]
%(name)s:%(levelname)s %(message)s

This will log all child loggers of sms which have levels of DEBUG or above to the syslog.

To use it in a Python application use this code:

import logging
import logging.config
logging.config.fileConfig('/home/sms/sms/logging_basic.ini')
log = logging.getLogger('sms.some_child')

You can then log results like this:

log.debug(result)

Implementing the Send SMS API

Let's start with the send SMS library. Create an sms.py file within the /home/sms/sms directory:

"""\
Simple module for sending SMS messages
"""
import logging
import logging.config
logging.config.fileConfig('/home/sms/sms/logging_basic.ini')
log = logging.getLogger('sms.sms')

import urllib

API_KEY = 'XXXXXXXXXXX'

def send_sms(to, message):
    log.debug("Sending SMS to %r with message %r", to, message)
    url = 'http://sms.alpha.orange-api.net/sms/sendSMS.xml'
    data = {'id': API_KEY, 'to':to, 'content':message}
    fp = urllib.urlopen(url, urllib.urlencode(data))
    result = fp.read()
    fp.close()
    log.debug("Sending SMS result was %r", result)
    return result

You'll need to replace all the XXX characters with the correct API key you get from the Orange control panel.

Implement the Web Server Part

Now we are ready to implement the webserver part. This is going to When a txt is sent to 967482 and starts with the keyword you've specified, Orange will call the URL you entered. For example. This text:

For 447980233595 Test

would result in this URL being called:

http://sms.3aims.com/retrieve?api=receivesms&from=447980233595&dateCreated=2008-05-18+22%3A20%3A10.863&content=For+07964763169+Test

Our code needs to handle this and then make a call to the API to send the txt on:

#! -*- coding: utf-8 -*-

import site
site.addsitedir('/home/sms/env/lib/python2.5')
site.addsitedir('/home/sms/sms')
site.addsitedir('/home/sms/env/lib/python2.5/site-packages')
from webob import Request, Response

# Set up logging
import logging
import logging.config
logging.config.fileConfig('/home/sms/sms/logging_basic.ini')
log = logging.getLogger('sms.handle')

from sms import send_sms
import webhelpers.mail as mail

def application(environ, start_response):
    log.debug('Message receive begun')
    request = Request(environ)
    response = Response()
    response.content_type = 'text/html'
    if request.path_info == '/retrieve':
        if request.params.get('api') != 'receivesms':
            log.error('Invalid API')
            response.body = 'Invalid API'
        elif not request.params.get('content'):
            log.error('No content')
            response.body = 'No content'
        elif not request.params.get('from'):
            log.error('No from')
            response.body = 'No from'
        else:
            content = request.params['content'].strip(' ')
            parts = content.split(' ')
            if len(parts) < 3:
                log.error('Unknown SMS structure')
                response.body = 'Unknown SMS structure'
            else:
                code = parts[0].lower()
                to = parts[1].lower()
                content = ' '.join(parts[2:])
                log.debug('Code: %r, To: %r, Content: %r', code, to, content)
                if code not in ['for']:
                    log.error('Received SMS with an unknown keyword %r', code)
                    response.body = 'Unknown SMS keyword'
                else:
                    if '@' in to:
                        # Treat as an email address
                        log.debug('Sending email to %r', to)
                        mail.send(
                            mail._prepare(
                                mail.plain(content),
                                from_name = '3aims SMS Gateway',
                                from_email = 'noreply@sms.3aims.com',
                                to=[to],
                                subject = 'SMS from %s'%request.params['from']
                            ),
                            sendmail = '/usr/sbin/sendmail'
                        )
                        log.debug('Sent email')
                        response.body = 'Sent email'
                    else:
                        if to.startswith('0'):
                            # Assume a UK number
                            to = '44'+to[1:]
                            log.debug('No country code, using %r', to)
                        from_ = 'From '+request.params['from']+'. '+content
                        log.debug('Sending SMS to %r from %r', to, from_)
                        response.body = send_sms(to, from_)
                        log.debug('Result: %r', response.body)
    else:
        response.status = '200 Not Found'
        response.body = 'Not Found'
    return response(environ, start_response)

Set up the Forwarding

When messages are sent to sms att sms.3aims.com we want them to piped to a program which sends them as an SMS message. This requires us to create a .forward file for the sms user to forward the email to the program.

As the sms user:

touch ~/.forward
chmod 0640 ~/.forward

Add the following content to the .forward file:

"|/home/sms/sms/receive.py"

We'll need the latest WebHelpers package:

cd ~/lib
hg clone http://pylonshq.com/hg/webhelpers
cd webhelpers
~/env/bin/python2.5 setup.py develop

Then create /home/sms/sms/receive.py with this content:

#!/home/sms/env/bin/python2.5

import logging
import logging.config
import sys
import webhelpers.mail as mail
from email.feedparser import FeedParser
from sms import send_sms

# Set up logging
logging.config.fileConfig('/home/sms/sms/logging_basic.ini')
log = logging.getLogger('sms.receive')

# Parse the message
f = FeedParser()
f.feed(sys.stdin.read())
message = f.close()

# Process the message
to = message['subject']
if not message.is_multipart():
    msg = message.get_payload()
else:
    raise Exception('Multipart messages not supported')

result = send_sms(to, msg)
log.debug(result)
log.debug(to)
log.debug(msg)

That's it. You can now send SMS's to email addresses, have emails forwarded to SMS and also have SMS's send to other mobile numbers all via your server.

With these basics in place the possibilities are fairly limitless.