Wednesday, February 26, 2014

WiFi Characteristics of mobile phones in screen lock mode

I have a Nexus 5 and I was wondering what power strategies/algorithms are used in it to save power when the screen is locked and the display is off. I wanted to have a practical first hand experience of seeing this happening.

So I pinged the phone from my computer while it was connected to my WiFi network. I did performed the process twice - once while the phone was active and I was switching between/using apps on it, and the second time when the screen was locked and the display was off.
I sent out 100 ping packets and here are the results:
Ping during locked screen with display off

Ping when the phone was active
Summary:

  • When screen was locked and display was off
    • Minimum = 8ms, Maximum = 866ms, Average = 295ms
    • Sporadic replies to ping, although no no packet was lost. Maybe the WiFi receiver was always on but the transmitter was only turned on periodically.
    • A very large variance in the round trip times.

  • When the phone was active
    • Minimum = 3ms, Maximum = 387ms, Average = 63ms
    • Quick replies. WiFi chipset must always be active.
    • Lesser variance in round trip times

Friday, February 21, 2014

Generic Netlink sockets - example code

If you want to use netlink as a userspace-kernelspace interface for your own non-networking custom use, make sure to go the Generic Netlink path - get a family id assigned and then used that to exchange messages between your userspace and kernelspace. That said, its better to just use Netlink Protocol Library Suite (libnl). If for some extremely compelling reason you can't, use libnl, here is some sample code to get you started.
Its based on Ariane Kellar's code from here. I have simplified and commented the userspace side code a lot. The kernel space code is mostly unchanged. The kernel side code required a minor change in genlmsg_unicast() call to ensure compatibility with the newer kernel versions.


nl_kern.c :

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
#include <net/genetlink.h>
#include <linux/module.h>
#include <linux/kernel.h>

//Code based on http://people.ee.ethz.ch/~arkeller/linux/multi/kernel_user_space_howto-3.html

/* attributes (variables):
 * the index in this enum is used as a reference for the type,
 * userspace application has to indicate the corresponding type
 * the policy is used for security considerations 
 */
enum {
 DOC_EXMPL_A_UNSPEC,
 DOC_EXMPL_A_MSG,
    __DOC_EXMPL_A_MAX,
};
#define DOC_EXMPL_A_MAX (__DOC_EXMPL_A_MAX - 1)

/* attribute policy: defines which attribute has which type (e.g int, char * etc)
 * possible values defined in net/netlink.h 
 */
static struct nla_policy doc_exmpl_genl_policy[DOC_EXMPL_A_MAX + 1] = {
 [DOC_EXMPL_A_MSG] = { .type = NLA_NUL_STRING },
};

#define VERSION_NR 1
//family definition
static struct genl_family doc_exmpl_gnl_family = {
 .id = GENL_ID_GENERATE,         //Genetlink should generate an id
 .hdrsize = 0,
 .name = "CONTROL_EXMPL",        //The name of this family, used by userspace application
 .version = VERSION_NR,          //Version number  
 .maxattr = DOC_EXMPL_A_MAX,
};

/* commands: enumeration of all commands (functions), 
 * used by userspace application to identify command to be executed
 */
enum {
 DOC_EXMPL_C_UNSPEC,
 DOC_EXMPL_C_ECHO,
 __DOC_EXMPL_C_MAX,
};
#define DOC_EXMPL_C_MAX (__DOC_EXMPL_C_MAX - 1)


//An echo command, receives a message, prints it and sends another message back
int doc_exmpl_echo(struct sk_buff *skb_2, struct genl_info *info) {
 struct nlattr *na;
 struct sk_buff *skb;
 int rc;
 void *msg_head;
 char * mydata;
 
 if (info == NULL) {
  goto out;
 }
  
    /* For each attribute there is an index in info->attrs which points to a nlattr structure
     * in this structure the data is given
     */
    na = info->attrs[DOC_EXMPL_A_MSG];
    if (na) {
  mydata = (char *)nla_data(na);
  if (mydata == NULL) {
   printk("error while receiving data\n");
  } else {
   printk("received: %s\n", mydata);
  }
 } else {
  printk("no info->attrs %i\n", DOC_EXMPL_A_MSG);
 }

    //Send a message back
    //Allocate some memory, since the size is not yet known use NLMSG_GOODSIZE
 skb = genlmsg_new(NLMSG_GOODSIZE, GFP_KERNEL);
 if (skb == NULL) {
  goto out;
 }

 //Create the message headers
    /* arguments of genlmsg_put: 
       struct sk_buff *, 
       int (sending) pid, 
       int sequence number, 
       struct genl_family *, 
       int flags, 
       u8 command index (why do we need this?)
    */
 msg_head = genlmsg_put(skb, 0, info->snd_seq+1, &doc_exmpl_gnl_family, 0, DOC_EXMPL_C_ECHO);
 if (msg_head == NULL) {
  rc = -ENOMEM;
  goto out;
 }
 //Add a DOC_EXMPL_A_MSG attribute (actual value to be sent)
 rc = nla_put_string(skb, DOC_EXMPL_A_MSG, "Hello World from kernel space");
 if (rc != 0) {
  goto out;
 }
 
 //Finalize the message
 genlmsg_end(skb, msg_head);

    //Send the message back
 rc = genlmsg_unicast(genl_info_net(info), skb,info->snd_pid );
 if (rc != 0) {
  goto out;
 }
 return 0;

 out:
 printk("An error occured in doc_exmpl_echo:\n");
 return 0;
}

//Commands: mapping between the command enumeration and the actual function
struct genl_ops doc_exmpl_gnl_ops_echo = {
 .cmd = DOC_EXMPL_C_ECHO,
 .flags = 0,
 .policy = doc_exmpl_genl_policy,
 .doit = doc_exmpl_echo,
 .dumpit = NULL,
};

static int __init gnKernel_init(void) {
 int rc;
 printk("Generic Netlink Example Module inserted.\n");
        
    //Register the new family
 rc = genl_register_family(&doc_exmpl_gnl_family);
 if (rc != 0) {
  goto failure;
 }
 //Register functions (commands) of the new family
 rc = genl_register_ops(&doc_exmpl_gnl_family, &doc_exmpl_gnl_ops_echo);
 if (rc != 0) {
  printk("Register ops: %i\n",rc);
  genl_unregister_family(&doc_exmpl_gnl_family);
  goto failure;
 }
 return 0; 
failure:
 printk("An error occured while inserting the generic netlink example module\n");
 return -1;
}

static void __exit gnKernel_exit(void) {
 int ret;
 printk("Generic Netlink Example Module unloaded.\n");
 
 //Unregister the functions
 ret = genl_unregister_ops(&doc_exmpl_gnl_family, &doc_exmpl_gnl_ops_echo);
 if(ret != 0) {
  printk("Unregister ops: %i\n",ret);
 }

    //Unregister the family
 ret = genl_unregister_family(&doc_exmpl_gnl_family);
 if(ret !=0) {
  printk("Unregister family %i\n",ret);
  return;
 }
}

module_init(gnKernel_init);
module_exit(gnKernel_exit);
MODULE_LICENSE("GPL");

     



nl_user.c :

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <unistd.h>
#include <poll.h>
#include <string.h>
#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <signal.h>

#include <linux/genetlink.h>

//Code based on http://people.ee.ethz.ch/~arkeller/linux/multi/kernel_user_space_howto-3.html

/* Generic macros for dealing with netlink sockets. Might be duplicated
 * elsewhere. It is recommended that commercial grade applications use
 * libnl or libnetlink and use the interfaces provided by the library
 */
#define GENLMSG_DATA(glh) ((void *)(NLMSG_DATA(glh) + GENL_HDRLEN))
#define GENLMSG_PAYLOAD(glh) (NLMSG_PAYLOAD(glh, 0) - GENL_HDRLEN)
#define NLA_DATA(na) ((void *)((char*)(na) + NLA_HDRLEN))

#define MESSAGE_TO_KERNEL "Hello World!"

//Variables used for netlink
int nl_fd;  //netlink socket's file descriptor
struct sockaddr_nl nl_address; //netlink socket address
int nl_family_id; //The family ID resolved by the netlink controller for this userspace program
int nl_rxtx_length; //Number of bytes sent or received via send() or recv()
struct nlattr *nl_na; //pointer to netlink attributes structure within the payload 
struct { //memory for netlink request and response messages - headers are included
    struct nlmsghdr n;
    struct genlmsghdr g;
    char buf[256];
} nl_request_msg, nl_response_msg;


int main(void) {
//Step 1: Open the socket. Note that protocol = NETLINK_GENERIC
    nl_fd = socket(AF_NETLINK, SOCK_RAW, NETLINK_GENERIC);
    if (nl_fd < 0) {
  perror("socket()");
  return -1;
    }

//Step 2: Bind the socket.
 memset(&nl_address, 0, sizeof(nl_address));
 nl_address.nl_family = AF_NETLINK;
 nl_address.nl_groups = 0;

 if (bind(nl_fd, (struct sockaddr *) &nl_address, sizeof(nl_address)) < 0) {
  perror("bind()");
  close(nl_fd);
  return -1;
 }

//Step 3. Resolve the family ID corresponding to the string "CONTROL_EXMPL"
    //Populate the netlink header
    nl_request_msg.n.nlmsg_type = GENL_ID_CTRL;
    nl_request_msg.n.nlmsg_flags = NLM_F_REQUEST;
    nl_request_msg.n.nlmsg_seq = 0;
    nl_request_msg.n.nlmsg_pid = getpid();
    nl_request_msg.n.nlmsg_len = NLMSG_LENGTH(GENL_HDRLEN);
    //Populate the payload's "family header" : which in our case is genlmsghdr
    nl_request_msg.g.cmd = CTRL_CMD_GETFAMILY;
    nl_request_msg.g.version = 0x1;
    //Populate the payload's "netlink attributes"
 nl_na = (struct nlattr *) GENLMSG_DATA(&nl_request_msg);
    nl_na->nla_type = CTRL_ATTR_FAMILY_NAME;
    nl_na->nla_len = strlen("CONTROL_EXMPL") + 1 + NLA_HDRLEN;
    strcpy(NLA_DATA(nl_na), "CONTROL_EXMPL"); //Family name length can be upto 16 chars including \0
    
    nl_request_msg.n.nlmsg_len += NLMSG_ALIGN(nl_na->nla_len);

 memset(&nl_address, 0, sizeof(nl_address));
 nl_address.nl_family = AF_NETLINK;

 //Send the family ID request message to the netlink controller
 nl_rxtx_length = sendto(nl_fd, (char *) &nl_request_msg, nl_request_msg.n.nlmsg_len,
  0, (struct sockaddr *) &nl_address, sizeof(nl_address));
 if (nl_rxtx_length != nl_request_msg.n.nlmsg_len) {
  perror("sendto()");
  close(nl_fd);
  return -1;
    }

 //Wait for the response message
    nl_rxtx_length = recv(nl_fd, &nl_response_msg, sizeof(nl_response_msg), 0);
    if (nl_rxtx_length < 0) {
        perror("recv()");
        return -1;
    }

    //Validate response message
    if (!NLMSG_OK((&nl_response_msg.n), nl_rxtx_length)) {
        fprintf(stderr, "family ID request : invalid message\n");
        return -1;
    }
    if (nl_response_msg.n.nlmsg_type == NLMSG_ERROR) { //error
        fprintf(stderr, "family ID request : receive error\n");
        return -1;
    }

    //Extract family ID
    nl_na = (struct nlattr *) GENLMSG_DATA(&nl_response_msg);
    nl_na = (struct nlattr *) ((char *) nl_na + NLA_ALIGN(nl_na->nla_len));
    if (nl_na->nla_type == CTRL_ATTR_FAMILY_ID) {
        nl_family_id = *(__u16 *) NLA_DATA(nl_na);
    }

//Step 4. Send own custom message
 memset(&nl_request_msg, 0, sizeof(nl_request_msg));
 memset(&nl_response_msg, 0, sizeof(nl_response_msg));

    nl_request_msg.n.nlmsg_len = NLMSG_LENGTH(GENL_HDRLEN);
    nl_request_msg.n.nlmsg_type = nl_family_id;
    nl_request_msg.n.nlmsg_flags = NLM_F_REQUEST;
    nl_request_msg.n.nlmsg_seq = 60;
    nl_request_msg.n.nlmsg_pid = getpid();
    nl_request_msg.g.cmd = 1; //corresponds to DOC_EXMPL_C_ECHO;
        
    nl_na = (struct nlattr *) GENLMSG_DATA(&nl_request_msg);
    nl_na->nla_type = 1; // corresponds to DOC_EXMPL_A_MSG
    nl_na->nla_len = sizeof(MESSAGE_TO_KERNEL)+NLA_HDRLEN; //Message length
    memcpy(NLA_DATA(nl_na), MESSAGE_TO_KERNEL, sizeof(MESSAGE_TO_KERNEL));
    nl_request_msg.n.nlmsg_len += NLMSG_ALIGN(nl_na->nla_len);

    memset(&nl_address, 0, sizeof(nl_address));
 nl_address.nl_family = AF_NETLINK;

 //Send the custom message
 nl_rxtx_length = sendto(nl_fd, (char *) &nl_request_msg, nl_request_msg.n.nlmsg_len,
  0, (struct sockaddr *) &nl_address, sizeof(nl_address));
 if (nl_rxtx_length != nl_request_msg.n.nlmsg_len) {
  perror("sendto()");
  close(nl_fd);
  return -1;
    }
    printf("Sent to kernel: %s\n",MESSAGE_TO_KERNEL);

    //Receive reply from kernel
    nl_rxtx_length = recv(nl_fd, &nl_response_msg, sizeof(nl_response_msg), 0);
    if (nl_rxtx_length < 0) {
        perror("recv()");
        return -1;
    }

 //Validate response message
    if (nl_response_msg.n.nlmsg_type == NLMSG_ERROR) { //Error
        printf("Error while receiving reply from kernel: NACK Received\n");
        close(nl_fd);
        return -1;
    }
    if (nl_rxtx_length < 0) {
        printf("Error while receiving reply from kernel\n");
        close(nl_fd);
        return -1;
    }
    if (!NLMSG_OK((&nl_response_msg.n), nl_rxtx_length)) {
        printf("Error while receiving reply from kernel: Invalid Message\n");
        close(nl_fd);
     return -1;
 }

    //Parse the reply message
    nl_rxtx_length = GENLMSG_PAYLOAD(&nl_response_msg.n);
    nl_na = (struct nlattr *) GENLMSG_DATA(&nl_response_msg);
    printf("Kernel replied: %s\n",(char *)NLA_DATA(nl_na));

//Step 5. Close the socket and quit
    close(nl_fd);
    return 0;
}


Makefile :

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
obj-m += nl_kern.o
nl_kern-objs := kern/nl_kern.o

all: kernel-module-uninstall kernel-clean-ring-buffer kernel-build kernel-clean-temporary kernel-module-install user-build
 @tput setaf 3
 @echo "    done: all"
 @tput sgr0
clean: kernel-module-uninstall kernel-clean user-clean
 @tput setaf 3
 @echo "    done: clean"
 @tput sgr0
 
 
 
kernel-build:
 @tput setaf 1
 @echo "    kernel-build"
 @tput sgr0
 make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules
kernel-clean:
 @tput setaf 1
 @echo "    kernel-clean"
 @tput sgr0
 make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean
kernel-clean-temporary:
 @tput setaf 1
 @echo "    kernel-clean-temporary"
 @tput sgr0
 -rm -rf *.o *~ core .depend .*.cmd *.mod.c .tmp_versions
 -rm -rf kern/*.o kern/*~ kern/core kern/.depend kern/.*.cmd kern/*.mod.c kern/.tmp_versions
 -rm -rf Module.symvers modules.order
kernel-module-install:
 @tput setaf 1
 @echo "    kernel-module-install"
 @tput sgr0
 -sudo insmod nl_kern.ko
kernel-module-uninstall:
 @tput setaf 1
 @echo "    kernel-module-uninstall"
 @tput sgr0
 -sudo rmmod nl_kern
kernel-clean-ring-buffer:
 @tput setaf 1
 @echo "    kernel-clean-ring-buffer"
 @tput sgr0
 sudo dmesg -c > /dev/null

 
 
user-build:
 @tput setaf 1
 @echo "    user-build"
 @tput sgr0
 gcc user/nl_user.c -o nl_user.out
user-clean:
 @tput setaf 1
 @echo "    user-clean"
 @tput sgr0
 rm -rf *.out

Keep the above source files organized in the folders like this if you want to use the above Makefile:

IoT Weather Station using TP-LINK TL-MR3020

What does it do?
Uploads Temperature, Humidity, Light and Pressure readings to Xively every minute

What is it made of?
What does it look like?
The USB Weather Board is installed outside in the balcony.
There are slots cut out in the box for exposing the sensors to the atmosphere.
The TL-MR3020 Pocket WiFi Router is mounted inside.
The power is drawn from the AC mains outlet meant for Air Conditioning.


Where are the reading being uploaded online?
Visit https://xively.com/feeds/1882537129 to have a look.
This is what the page looks like:


How much did it cost? Can the cost be reduced?
Refer to the blog post here for the cost of the various components involved:

The USB Weather Board from Sparkfun used to cost USD 125.
I had bought it locally from http://www.tenettech.com for INR 10,287.
The USB Weather Board is discontinued now and is now replaced with Weather Shield.
The Weather Shield when paired with an Arduino UNO R3 would function exactly like the older USB Weather Board.
This new combo of Weather Shield +Arduino UNO R3 costs lesser than the original USB Weather Board.

How did you build it?
  1. Get a TL-MR3020 router and connected it to your laptop using a straight LAN cable. 
  2. Flash it with OpenWRT. Look here to know how to do that. Make sure to follow the "Method Using Web GUI". Use only Ethernet while flashing the router - do not use WiFi.
  3. Once you are done flashing, the first thing you will have to is to set the default password for WebGUI. Punch in 192.168.1.1 in your browser and OpenWRT will ask you to set the password.
    Setting the password for OpenWRT
  4. Next configure the router to act as a WiFi client instead of a WiFi access point. Navigate to Network>WiFi on the OpenWRT Web GUI. You will note that there is an entry "Generic MAC80211 802.11bgn (radio0)". This is the WiFi Access Point entry. Click the Remove button and delete this network.
    Delete the entry for WiFi Access Point
  5. Now while on the same page, click on "Scan" and scroll through the list of network and when you find the entry for your home network, click "Join Network". You will be asked to enter the password.
    Join your home WiFi network. MR3020 will not become a client of your
    home WiFi Router though which it will access the internet
  6. Modify the firewall rules. Navigate to Network>Firewall on the Web GUI and set all the Inputs to accept, all the Outputs to accept and all Forward to reject.
    Firewall rules: Set all Inputs and Outputs to Accept
  7. At this point you can unplug the ethernet cable from your M3020 and start accessing it over WiFi. Take note that the IP address for the MR3020 will be different now - it will be the one assigned by your home WiFi Router. Use this new IP address to access the MR3020's OpenWRT Web GUI.
  8. The MR3020 has 4MB of flash memory. After flashing the router with OpenWRT, only 840 kilobytes are left. You can check this under System>Software on the Web GUI. This page will allow you to install additional software that we may require. But the problem is all the software that we need will require more than 840KB of space.
    Only 840 KB are available on MR3020's internal flash for our use.
  9. So what we need to do is connect an external USB flash drive and mount the root filesystem on it. This will give us access of multiple gigabytes of space and we can install all the software we want on the MR3020. The MR3020 only has the one USB port. So we need to use a small USB hub which will allow us to connect the USB flash drive as well as the USB Weather Board to it. Before we move the rootfs to external USB drive, we need to install kernel mode drivers for USB hub and USB flash drive. The next steps will show you how to do that. Do not connect the USB hub and the flash drive to MR3020 yet.
    MR3020 with USB Hub, Flash Drive and wire going to the Weather Board.
  10. Download PuTTY on your laptop and connect to the MR3020 to gain console access. Start PuTTY, punch in the MR3020's IP address and click open.
    Gaining console access using PuTTY
  11. While connecting, PuTTY main complain about the keys not being present in the cache, just ignore the message by pressing yes.
  12. When the console comes up, you will be asked for a login name. Punch in "root", next punch in the password - its the same that you initially set on the Web GUI.
    Console access to MR3020 using PuTTY
  13. Now we need to fetch some packages from the internet and install it. We could have done it through System>Software from the Web GUI but this method is better and faster. Issue the command
    opkg update
    to update the package list. Next issue the following command to install the drivers:
    opkg install kmod-usb-storage block-mount kmod-fs-ext4 kmod-scsi-core
    USB Drivers Installed
  14. Now before we connect the flash drive and USB hub to the MR3020, we need to format the flash drive. You will need a Linux computer for this. I run Ubuntu using VirtualBox on my Windows Laptop and used that to format the flash drive. The filesystem I chose was ext4.
    Formatting the USB flash driver before use with the MR3020
  15. Next, turn off your MR3020, attach the USB hub and the flash driver and reconnect to its console using PuTTY.
  16. We need to check the path where MR3020 has mounted the flash drive. Execute the command
    ls /dev/sd*
    to know the path. In my case this was /dev/sdb1. In your case it could be /dev/sda1. Use this path in the following steps wherever required.
  17. The remaining steps required to move the rootfs to the flash drive have been taken from this page here. Mount the USB stick and copy the flash /overlay to the USB stick by executing the following four commands in succession:
    1. mkdir -p /mnt/usb
    2. mount -t vfat /dev/sdb1 /mnt/usb
    3. tar -C /overlay -cvf - . | tar -C /mnt/usb -xvf -
    4. vi /etc/config/fstab
  18. The last of the above four commands will open a text editor. Press I to insert and modify the file. The should look like this:
    config 'mount'
            option target   /overlay
            option device   /dev/sdb1
            option fstype   ext4
            option options  rw,sync
            option enabled  1
            option enabled_fsck 0
  19. After editing the file press Escape on your keyboard followed by colon (:) and then type "wq!" and press Enter to save the file.
  20. Reboot the router and check if you have gained the extra space of the USB flash drive using the command df -h or via the Web GUI.
    Confirming that the rootfs now has 3.4 GB of Space (used a 4GB Flash drive)
    via the SSH console using the command df -h

    Confirming the space gain using Web GUI
  21. Next we need some additional software - utilities (curl) for communicating with Xively and drivers for FT232 USB-UART chip present on the USB Weather Board. To install all this, issue the following command via the PuTTY Console:
    opkg install kmod-usb-serial-ftdi curl coreutils-stty usbutilsAlternatively, if you dont want to use the console, you can install these packages one by one using the Web GUI via System>Software.
  22. Now before you connect the USB Weather Board to the MR3020, connect it to a laptop and configure it with the following settings (Refer to the PDF user manual of the weather board to know how to do this):
    1. Data format: CSV
    2. Units: SI
    3. Pressure: Absolute
    4. Baud: 9600
    5. Sample Rate: 5 seconds
      I enclosed the Sparkfun USB Weather Board in a small Lunch Box.
      Made holes in it so that the sensors stay exposed to the environment.
      Used blue-tac to seal the sides of the cut out hole.
  23. Turn off the MR3020 and plug in the USB Weather Board. Turn the router on and check if the FT232 Chip was detected and enumerated correctly. Use the command ls /dev and check if a new device "ttyUSB0" appears there
    Checking if /dev/ttyUSB0 appears
    The lsusb commands can also be used to check the list of devices
    connected to MR3020 over USB
  24. Before the next step, you need to create an account at www.xively.com and create a device there. Note down the API key and the feed ID.
  25. Copy the following code into a text file on your Windows Laptop and and save it as "weather.sh". Modify the API_KEY and the FEED_ID and change them to the one assigned to your own Xively account/device while doing so.
      1
      2
      3
      4
      5
      6
      7
      8
      9
     10
     11
     12
     13
     14
     15
     16
     17
     18
     19
     20
     21
     22
     23
     24
     25
     26
     27
     28
     29
     30
     31
     32
     33
     34
     35
     36
     37
     38
     39
     40
     41
     42
     43
     44
     45
     46
     47
     48
     49
     50
     51
     52
     53
     54
     55
     56
     57
     58
     59
     60
     61
     62
     63
     64
     65
     66
     67
     68
     69
     70
     71
     72
     73
     74
     75
     76
     77
     78
     79
     80
     81
     82
     83
     84
     85
     86
     87
     88
     89
     90
     91
     92
     93
     94
     95
     96
     97
     98
     99
    100
    101
    102
    103
    104
    105
    106
    107
    108
    109
    # Xively Configuration:
    API_KEY="z0K7TXFxNWTCdfdsfdsfdsfcLLRxrsktnYgdfdfdg2Rx4S"
    FEED_ID="1884343429"
    
    
    echo 'Make sure the Sparkfun Weather board is configured with the following settings:'
    echo 'Data format: CSV | Units: SI | Pressure: Absolute | baud: 9600 | Sample Rate: 5 seconds'
    echo '---------------------------------------------------------------------------------------'
    echo ''
    
    
    LOOP_COUNTER=0
    
    # Serial Port setting
    stty -F /dev/ttyUSB0 9600 cs8 -cstopb
    
    
    
    # Set the internal field separator to comma
    OLDIFS=$IFS
    IFS=,
    
    # Loop
    while read header_char temp_SHT15 humidity_SHT15 dewpoint_SHT15 presure_BMP085 light_TEMT6000 wind_speed wind_direction rainfall battery trailing_char;
    do
     let "LOOP_COUNTER += 1"
     if [ "$LOOP_COUNTER" == "10" ]; then
      logger "Loop Counter error! exiting"
      echo "Loop Counter error! exiting"
      break
     fi
     if [ "$header_char" != "$" ]; then
      continue
     fi
     
        echo " "
     echo "Header character: $header_char"
     echo "SHT15 temperature: $temp_SHT15 deg C"
     echo "SHT15 humidity: $humidity_SHT15 %"
     echo "SHT15 dewpoint: $dewpoint_SHT15 deg C"
     echo "BMP085 pressure (absolute): $presure_BMP085 mbar"
     echo "TEMT6000 light: $light_TEMT6000 %"
     echo "Weather meters wind speed: $wind_speed m/s"
     echo "Weather meters wind direction: $wind_direction degrees"
     echo "Weather meters rainfall: $rainfall mm"
     echo "Battery Voltage: $battery Volts"
     echo "Trailing Character: $trailing_char"
     
     # Restore field separator
     IFS=$OLDIFS
     
     DATA_JSON="{"
     DATA_JSON="$DATA_JSON"$'\n'"  \"version\":\"1.0.0\","
     DATA_JSON="$DATA_JSON"$'\n'"   \"datastreams\" : [ {"
     DATA_JSON="$DATA_JSON"$'\n'"      \"id\" : \"Temperature\","
     DATA_JSON="$DATA_JSON"$'\n'"      \"current_value\" : \"$temp_SHT15\""
     DATA_JSON="$DATA_JSON"$'\n'"    },"
     DATA_JSON="$DATA_JSON"$'\n'"    { \"id\" : \"Humidity\","
     DATA_JSON="$DATA_JSON"$'\n'"      \"current_value\" : \"$humidity_SHT15\""
     DATA_JSON="$DATA_JSON"$'\n'"    },"
     DATA_JSON="$DATA_JSON"$'\n'"    { \"id\" : \"Light\","
     DATA_JSON="$DATA_JSON"$'\n'"      \"current_value\" : \"$light_TEMT6000\""
     DATA_JSON="$DATA_JSON"$'\n'"    },"
     DATA_JSON="$DATA_JSON"$'\n'"    { \"id\" : \"Pressure\","
     DATA_JSON="$DATA_JSON"$'\n'"      \"current_value\" : \"$presure_BMP085\""
     DATA_JSON="$DATA_JSON"$'\n'"    }"
     DATA_JSON="$DATA_JSON"$'\n'"  ]"
     DATA_JSON="$DATA_JSON"$'\n'"}"
    
     curl --max-time 5 \
      --request PUT \
      --data "$DATA_JSON" \
      --header "X-ApiKey: $API_KEY" \
      --verbose \
      http://api.xively.com/v2/feeds/"$FEED_ID"
     
     #if [ "$?" != 0 ]; then
      echo "Xively PUT Success! (temp=$temp_SHT15, humidity=$humidity_SHT15, light=$light_TEMT6000, pressure=$presure_BMP085)"  
      logger "Xively PUT Success! (temp=$temp_SHT15, humidity=$humidity_SHT15, light=$light_TEMT6000, pressure=$presure_BMP085)"  
     #fi
     break
    done < /dev/ttyUSB0
    
    echo "Done!"
    
    # References:
    # Arduino OpenWRT USB connection
    # http://lectroleevin.wordpress.com/2011/10/26/arduino-openwrt-usb-connection/
    
    # Bash read from ttyUSB0 and send to URL
    # http://stackoverflow.com/questions/4942502/bash-read-from-ttyusb0-and-send-to-url
    
    # Change UART serial port speed (baud rate) on OpenWrt
    # http://wiki.openwrt.org/doc/recipes/serialbaudratespeed
    
    # Looping through the content of a file in Bash?
    # http://stackoverflow.com/questions/1521462/looping-through-the-content-of-a-file-in-bash
    
    # Bash Read Comma Separated CVS File
    # http://www.cyberciti.biz/faq/unix-linux-bash-read-comma-separated-cvsfile/
    
    # OpenWRT on TP-Link MR3020 Guide w/USB Support
    # http://store.jpgottech.com/support/tp-link-mr3020-openwrt-flashing-guide/
    
    # Rootfs on External Storage (extroot)
    # http://wiki.openwrt.org/doc/howto/extroot
    
    # HowTo: Add Jobs To cron Under Linux or UNIX?
    # http://www.cyberciti.biz/faq/how-do-i-add-jobs-to-cron-under-linux-or-unix-oses/
    
  26. Important Step: You will need to convert the line ending from Windows format to Linux Format. Refer to this blog post to know how to do this. 
  27. Copy the file weather.sh to the MR3020 using WinSCP AND set the executable permissions for it. WinSCP works just like PuTTY but instead of gaining console access, it is used to copy files to the target. Copy the weather.sh file to /root.
    Start WinSCP, set the file protocol to SCP, enter the IP address.
    Enter the user name as "root" and password that you had set
    through the Web GUI and click Login.
    Copy the Weather.sh to /root on the MR3020
    Right click on the weather.sh file on MR3020 in WinSCP, select properties.
    Check the boxes next to the X bits and click OK.
  28. The last step requires you to set the script as a cron job. Open the Web GUI and set the script to execute every minute. Do this via System > Scheduled Tasks. Punch in the following line in the text box, save it and reboot your MR3020 and it will start uploading the sensor readings to your Xively account every minute:
    * * * * * /root/weather.sh

    Adding a cron job to MR3020
My Weather Station has been running reliably for more than 2 months now. It has been uploading the reading every minute. You can look at the feed here https://xively.com/feeds/1882537129
You can see the graphs for data accumulated during the last 3 month period.




Adding entry for the script to the cron tab ensures that it gets back to work properly and reliably even after power cycles and power failures.

Update 2014-02-27 :
I have been using the mains socket meant for air conditioning to power the weather station for the past 3 months. Now that summer is almost here (and so are regular power cuts), I decided to switch to the phase that is routed through the Uninterrupted Power Supply. I took the opportunity to mount everything in a single new box. Here is what it looks like now:



Update 2014-10-16:
The weather station is 10 months old now (I put it together in December 2013)  and since the last update, I encountered a few more problems which I solved like so:
The MR3020 and the Sparkfun weather board are now enclosed in separate lunchboxes. Notice the Ethernet cable coming out of the box on the left. I now use Ethernet instead of WiFi to provide internet access to MR3020. I already had a spare RJ45 on the switchboard nearby in the room. I had asked the builder to route CAT5e cables through the concealed wiring along with the power cables while the building was being constructed. 

The previous lunchbox now has a hole through which the 4GB flash drive sticks out and dissipates all its heat. The USB flash drive is the metal thingy below the blue colored USB hub.
  1. Connectivity Issues
    So I noticed that MR3020 started losing WiFi connection to my home router and then it would stay unconnected for days until I happened to check the Xively feed followed by power cycling the MR3020. I investigated the problem by connecting my laptop as an Ethernet client to MR3020 while it tried to connecting to my home WiFi router and observing the syslogs. The logs showed that MR3020 wasn't able to associate and stay associated with my home router and messages similar to the ones like this kept being spewed out into the syslog.

    I came up with the following reasons either or all of which may have contributed to this problem.
    1. I had relocated my home router to a more central location within my home. This meant moving it further away from the weather station thereby causing signal degradation
    2. The daily exposure to the elements (daytime heat and night time cold) may have caused the RF front-end components in MR3020 to drift causing the RF section to go a bit out of tuning.
    3. As more and more tenants shifted into the various apartments in and around my society, they started installing their own WiFi Routers causing crowding within the spectrum thereby causing further deterioration of the signal.

      To solve these issues, I tried switching WiFi channel to the lesser crowded one but that didn't work. Ultimately I reconfigured the MR3020 - disabled its WiFi and switched to using Ethernet as the interface for connecting to the internet. To ensure further stability, I manually assigned a static local IP address to it - did not want to depend on DHCP from my home router.

      I also configured MR3020 to send syslog messages to papertrailapp.com so that I could remotely view the syslog messages from it. Just connect to your MR3020 using PuTTY and follow the instructions given at https://papertrailapp.com/systems/setup for
      syslog-ng.conf
    4. WiFi Analyzer app on my Nexus 5 showed how crowded the 2.4 GHz spectrum had gotten  
       
      Paper Trail app lets me keep track of syslog messages from all my linux based IoT devices
    1. Readings getting affected by heating
      MR3020 and the USB flash drive generate a lot of heat when powered. This was causing inaccuracy in the temperature readings taken by the Sparkfun weather board which itself I had mounted in the same enclosure.

      To fix that problem, I removed the Sparkfun weather board and mounted it in a separate enclosure of its own isolating it from the heat generating components. I used hot glue to stick this new enclosure to the previous one which still housed the MR3020 (and USB flash drive and power supply)

      I also made a hole in the previous enclosure housing the MR3020 such that the 4GB flash drive could stick out of that hole and dissipate its heat. The flash drive generated the most heat and this allowed it to stay cool.

       
    2. Power Line Communications
      Most of the switch boards in my house have a RJ45 connector which provide a wired connection to my home router (via a 8 port switch). I had asked the builder to route CAT5e cables through the concealed wiring along with the power cables while the building was being constructed. In case you don't have an easy and non-ugly way to route a CAT5e cable to the router, you can use power line communications - using power cables to transfer ethernet frames between two points. The following devices which used in pairs allows you to turn power outlet into ethernet jacks. I have tested both of them and they work reliably
    COMFAST CF-WP200M 200Mbps RJ45 Mini HomePlug AV
    Install one near your weather station and connect it to the MR3020 using a CAT5 cable.
    Install the other one near your home router and stick a CAT5 cable between them.
    And your MR3020 shall have internet. no configuration required.


    Linksys PLWK400
    These are more expensive ones because they allow you to
    extend the range of your WiFi network using power line communication.
    They also double up as point to point Ethernet links like the COMFAST ones.
    I modified my Weather Station to use the COMFAST PLC modules when the WiFi
    link started failing due to spectrum overcrowding. The COMFAST modules worked well for a few week,
    but afterwards they too started glitching and had to be rebooted every now and then.
    I finally did away with Powerline as well as WiFi and installed a CAT5 wire from my Router to the weather station.
    It hasn't glitched even once since then. 
    Xively has since changed it business model and is no longer favourable for IoT DIYers. Please consider using www.thingspeak.com for your future projects. ThingSpeak is simpler to use and has a straightforward interface.

    Final revision with Ethernet cable from my WiFi router running into the Weather Station.
    The Ethernet cable and power cable are both enclosed in the plastic conduits.