How to add support for a new device

How to support a new device

All devices should inherit from WCDMAWrapper in order to override its base methods. It accepts two argument for its constructor, an instance or subclass of DevicePlugin and the udi of the device to use.

A DevicePlugin contains all the necessary information to speak with the device: a data port, a control port (if exists), a baudrate, etc. It also has the following attributes and members that you can customise for your device plugin:

• custom: An instance or subclass of wader.common.hardware.base.WCDMACustomizer. See bellow.
• sim_klass: An instance or subclass of wader.common.sim.SIMBaseClass.
• baudrate: At what speed are we going to talk with this device (default 115200).
• __remote_name__: As some devices share the same vendor and product ids, we will issue an AT+GMR command right at the beginning to find out the real device model. Set this attribute to whatever your device replies to the AT+GMR command.
• mapping: A dictionary that, when is not empty, means that that particular combination of vendor and product ids is shared between several models from the same company. As the ids are the same, the only way to differentiate them is issuing an AT+GMR command to get the device model. This dict must have an entry for each model and a default entry that will be used in cases where we do not know about the card model. For more information about the mapping dictionary, have a look at the module wader.common.plugins.huawei_exxx.

The object wader.common.hardware.base.WCDMACustomizer acts as a container for all the device-specific customizations such as:

• wrapper_klass: Specifies the class that will be used to wrap the Device object. Used in situations where the default commands supplied by wader.common.middleware.WCDMAWrapper are not enough: (i.e. devices with “interesting” firmwares). Some of its commands may need an special parsing or an specific workaround.
• exporter_klass: Specifies the class that will export the wrapper methods over DBus.
• State Machines: Each device its a world on its own, and even though they are supposed to support the relevant GSM and 3GPP standards, some devices prefer to differ from them. The custom object contains references to the state machines that the device should use, (on situations where it applies, such as with WCDMA devices of course):
  • auth_klass: The state machine used to authenticate against the device, default is wader.common.statem.auth.AuthStateMachine.
  • netr_klass: The state machine used to register on the network, default is wader.common.statem.networkreg.NetworkRegistrationStateMachine.
  • simp_klass: The simple state machine specified by ModemManager v0.2. This state machine basically comprises the previous two on a single (and simpler) one.
• async_regexp: regular expression object that will match whatever pattern of unsolicited notifications the given device sends us.
• signal_translations: Dictionary of tuples, each tuple has two members: the first is the signal id and the second is a function that will translate the signal arguments and the signal to the internal representation that Wader uses. You can find some sample code in the huawei module.
• band_dict: Dictionary with the AT strings necessary to switch the band technology that the device uses. The number of possible bands is limited by the device itself and the modifications that the manufacturer might have done to the device’s firmware. The reference band dictionary is in the hardware module.
• conn_dict: Dictionary with 5 items, each one defines the AT string that must be sent to the device in order to configure the connection mode preferences (Gprs only, 3G preferred, any, etc.) This dictionaries can be shared most of the time between different models from the same manufacturer.
• cmd_dict: Dictionary with information about how each command should be processed. cmd_dict most of the time will be a shallow copy of the command dict with minor modifications about how a particular command is processed on the given device.
• device_capabilities: List with all the unsolicited notifications that this device will send us. If the device sends us every RSSI change that detects, we don’t need to poll manually the device for that information.

Overview of a simple DevicePlugin

Lets have a look at the NovatelXU870 plugin:

import serial

from wader.common import consts
from wader.common.hardware.novatel import (NovatelWCDMADevicePlugin,
                                           NovatelWCDMACustomizer,
                                           NOVATEL_BAND_DICT)
from wader.common.hardware.base import build_band_dict


class NovatelXU870Customizer(NovatelWCDMACustomizer):
    """
    :class:`~wader.common.hardware.novatel.NovatelWCDMACustomizer` for XU870
    """

    # Supported bands (from Novatel docs)
    # GSM/GPRS
    #    GSM 850             824 -894MHz
    #    EGSM 900            880-960MHz
    #    DCS 1800            1710-1880MHz
    #    PCS 1900            1850-1990MHz
    # WCDMA
    #    UMTS 850 (Band V)   824 -894MHz
    #    UMTS 1900 (Band II) 1850-1990MHz
    #    UMTS 2100 (Band I)  1920-2170MHz

    band_dict = build_band_dict(
                  NOVATEL_BAND_DICT,
                  [consts.MM_NETWORK_BAND_ANY,

                   consts.MM_NETWORK_BAND_G850,
                   consts.MM_NETWORK_BAND_EGSM,
                   consts.MM_NETWORK_BAND_DCS,
                   consts.MM_NETWORK_BAND_PCS,

                   consts.MM_NETWORK_BAND_U850,
                   # XXX: Novatel docs show UMTS 1900 (Band II)
                   # but consts.py has this as UMTS 1900 Class IX
                   consts.MM_NETWORK_BAND_U1900,
                   consts.MM_NETWORK_BAND_U2100])


class NovatelXU870(NovatelWCDMADevicePlugin):
    """:class:`~wader.common.plugin.DevicePlugin` for Novatel's XU870"""
    name = "Novatel XU870"
    version = "0.1"
    author = u"Pablo Martí"
    custom = NovatelXU870Customizer()

    __remote_name__ = "Merlin XU870 ExpressCard"

    __properties__ = {
        'ID_VENDOR_ID': [0x1410],
        'ID_MODEL_ID': [0x1430],
    }

    def preprobe_init(self, ports, info):
        # Novatel secondary port needs to be flipped from DM to AT mode
        # before it will answer our AT queries. So the primary port
        # needs this string first or auto detection of ctrl port fails.
        # Note: Early models/firmware were DM only
        ser = serial.Serial(ports[0], timeout=1)
        ser.write('AT$NWDMAT=1\r\n')
        ser.close()

novatelxu870 = NovatelXU870()

In an ideal world, devices have a unique vendor and product id tuple, they conform to the relevant CDMA or WCDMA specs, and that’s it. The device is identified by its vendor and product ids and double-checked with its __remote_name__ attribute (the response to an AT+GMR command). This vendor and product id tuple will usually use the usb bus, however some devices might end up attached to the pci or pcmcia buses. The last line in the plugin will create an instance of the plugin in wader’s plugin system -otherwise it will not be found!.

Overview of a relatively simple DevicePlugin

Take for example the HuaweiE620 class:

import re

from wader.common.hardware.huawei import (HuaweiWCDMADevicePlugin,
                                          HuaweiWCDMACustomizer)
from wader.common.command import build_cmd_dict

E620_CMD_DICT = HuaweiWCDMACustomizer.cmd_dict.copy()
E620_CMD_DICT['get_roaming_ids'] = build_cmd_dict(re.compile(
                                    """
                                    \r\n
                                    \+CPOL:\s(?P<index>\d+),"(?P<netid>\d+)"
                                    """, re.VERBOSE))


class HuaweiE620Customizer(HuaweiWCDMACustomizer):
    cmd_dict = E620_CMD_DICT


class HuaweiE620(HuaweiWCDMADevicePlugin):
    """:class:`~wader.common.plugin.DevicePlugin` for Huawei's E620"""
    name = "Huawei E620"
    version = "0.1"
    author = u"Pablo Martí"
    custom = HuaweiE620Customizer()

    __remote_name__ = "E620"

    __properties__ = {
        'usb_device.vendor_id': [0x12d1],
        'usb_device.product_id': [0x1001],
    }

The E620 plugin is identical to the XU870 except one small difference regarding the parsing of the get_roaming_ids command. The E620 omits some information that other devices do output, and the regular expression object that parses it has to be updated. We get a new copy of the cmd_dict dictionary attribute and modify it with the new regexp the get_roaming_ids entry. The new cmd_dict is specified in its Customizer object.

Overview of a not so simple DevicePlugin

from wader.common.hardware.huawei import (HuaweiWCDMADevicePlugin,
                                          HuaweiSIMClass)

class HuaweiE220SIMClass(HuaweiSIMClass):
    """Huawei E220 SIM Class"""
    def __init__(self, sconn):
        super(HuaweiE220SIMClass, self).__init__(sconn)

    def initialize(self, set_encoding=False):
        d = super(HuaweiE220SIMClass, self).initialize(set_encoding)
        def init_cb(size):
            self.sconn.get_smsc()
            # before switching to UCS2, we need to get once the SMSC number
            # otherwise as soon as we send a SMS, the device would reset
            # as if it had been unplugged and replugged to the system
            def process_charset(charset):
                """
                Do not set charset to UCS2 if is not necessary, returns size
                """
                if charset == "UCS2":
                    return size
                else:
                    d3 = self.sconn.set_charset("UCS2")
                    d3.addCallback(lambda ignored: size)
                    return d3

            d2 = self.sconn.get_charset()
            d2.addCallback(process_charset)
            return d2

        d.addCallback(init_cb)
        return d


class HuaweiE220(HuaweiWCDMADevicePlugin):
    """:class:`~wader.common.plugin.DevicePlugin` for Huawei's E220"""
    name = "Huawei E220"
    version = "0.1"
    author = u"Pablo Martí"
    sim_klass = HuaweiE220SIMClass

    __remote_name__ = "E220"

    __properties__ = {
        'usb_device.vendor_id': [0x12d1],
        'usb_device.product_id': [0x1003, 0x1004],
    }

Huawei’s E220, despite sharing its manufacturer with the E620, has a couple of minor differences that deserve some explanation. There’s a bug in its firmware that will reset the device if you ask its SMSC. The workaround is to get once the SMSC before switching to UCS2, you’d be amazed of how long took me to discover the fix. The second difference with the E620 is that the E220 can have several product_ids, thus its product_id list has two elements.

Overview of a complex DevicePlugin

from twisted.python import log

from wader.common.hardware.option import (OptionWCDMADevicePlugin,
                                        OptionWCDMACustomizer)
from wader.common.sim import SIMBaseClass
from wader.common.statem.auth import AuthStateMachine
from wader.contrib.modal import mode


class OptionColtAuthStateMachine(AuthStateMachine):
    """
    Custom AuthStateMachine for Option Colt

    This device has a rather buggy firmware that yields all sort of
    weird errors. For example, if PIN authentication is disabled on the SIM
    and you issue an AT+CPIN? command, it will reply with a +CPIN: SIM PUK2
    """
    pin_needed_status = AuthStateMachine.pin_needed_status
    puk_needed_status = AuthStateMachine.puk_needed_status
    puk2_needed_status = AuthStateMachine.puk2_needed_status

    class get_pin_status(mode):
        """
        Returns the authentication status

        The SIM can be in one of the following states:

        - SIM is ready (already authenticated, or PIN disabled)
        - PIN is needed
        - PIN2 is needed (not handled)
        - PUK is needed
        - PUK2 is needed
        - SIM is not inserted
        - SIM's firmware error
        """

        def __enter__(self):
            pass

        def __exit__(self):
            pass

        def do_next(self):
            log.msg("Instantiating get_pin_status mode....")
            d = self.device.sconn.get_pin_status()
            d.addCallback(self.get_pin_status_cb)
            d.addErrback(self.sim_failure_eb)
            d.addErrback(self.sim_busy_eb)
            d.addErrback(self.sim_no_present_eb)
            d.addErrback(log.err)


class OptionColtSIMClass(SIMBaseClass):
    """Option Colt SIM Class"""

    def __init__(self, sconn):
        super(OptionColtSIMClass, self).__init__(sconn)

    def initialize(self, set_encoding=False):
        self.charset = 'IRA'
        d = super(OptionColtSIMClass, self).initialize(set_encoding)
        d.addCallback(self.set_size)
        return d


class OptionColtCustomizer(OptionWCDMACustomizer):
    """:class:`~wader.common.hardware.Customizer` for Option Colt"""
    auth_klass = OptionColtAuthStateMachine


class OptionColt(OptionWCDMADevicePlugin):
    """:class:`~wader.common.plugin.DevicePlugin` for Option Colt"""
    name = "Option Colt"
    version = "0.1"
    author = u"Pablo Martí"
    custom = OptionColtCustomizer()
    sim_klass = OptionColtSIMClass

    __remote_name__ = "129"

    __properties__ = {
        'ID_VENDOR_ID': [0x0af0],
        'ID_MODEL_ID': [0x5000],
    }


optioncolt = OptionColt()

This data card is the buggiest card we’ve found so far, and has proven to be an excellent challenge for the extensibility and granularity of our plugin system. Basically we’ve found the following bugs on the card’s firmware:

• If PIN authentication is disabled and you issue an AT+CPIN?, the card will reply with a +CPIN: SIM PUK2.
• Don’t ask me why, but AT+CPBR=1,250 does not work once the application is running. I have tried replacing the command with an equivalent one (AT+CPBF=”“) without luck.

So we had to modify the AuthStateMachine for this particular device and its cmd_dict.