Wednesday, December 16, 2015

OpenWrt 15.05 Chaos Calmer on TL MR3020 with rootfs on external usb drive

The latest version of OpenWrt is quite bulky. Although it feats neatly within TL-MR3020's 4MB flash, doesn't leave enough space to install the modules required to shift the root file system to an external drive.

My TL-MR3020 with FT232RL board to access its serial console
and a USB hub with a flash drive in one of its slots.

In previous versions of OpenWRT, you had 840KB of space left on internal 4MB flash to install the essential kernel modules for usb, usb mass storage and ext4 which are all essential for mounting USB flash drive. My Weather Station used TL-MR3020 flashed with OpenWrt 12.09 and used this methods.
In the latest version just about a few hundred kilobytes are left after flashing the image which aren't enough to install the modules required for mounting USB flash drives. So we have to resort to the following steps:


  1. Open up your TL-MR3020 and solder in a USB-UART (FT232RL) board to gain access to its serial port.
  2. Install the toolchain for compiling OpenWRT on your 64-bit Ubuntu Laptop
  3. Compile OpenWrt for TL-MR3020 on your ubuntu laptop. While doing so, we will include the modules required for mounting flash drives and leave out the bulky modules like LuCI web interface that can be installed later on the rootfs which has been relocated to USB flash drive)
  4. Connect your TL-MR3020 to your laptop using Ethernet cable and flash the newly compiled image
  5. Disconnect the ethernet cable between TL-MR3020 and your laptop and connect a USB cable to the soldered USB-UART board to gain access to TL-MR3020's Serial console
  6. Issue commands to TL-MR3020 using PuTTY over Serial and configure it to connect to your WiFi router as client.
  7. Issue commands to TL-MR3020 using PuTTY (over serial or over WiFi) and configure it to mount external USB flash drive as rootfs.
  8. Issue commands to TL-MR3020 using PuTTY (over serial or over WiFi) and configure it to install LuCI and other essential modules that had been left behind when you compiled the image for OpenWrt.
  9. Done. Now you can access your router using your browser as well.
Details:

  1. Open up your TL-MR3020 and solder in a USB-UART (FT232RL) board to gain access to its serial port. To do this refer to this image for the pin out. (You only need access to TX, RX and GND):
    Note the location of RX, TX and GND.
    Connect this to a SparkFun FTDI Basic Breakout - 3.3V Board
    Make sure to use only the 3.3V FTDI board
    Make sure you connect TX on TL-MR3020 to RX on FT232RL and
    RX on TL-MR3020 to TX on FT232RL.
    GND on MR3020 will connect to GND on FT232RL
  2. Install the OpenWrt toolchain on your 64-bit Ubuntu laptop.
    Instructions to do this are in my blog post here. The blog post covers how to install toolchain for Attitude Adjustment - a previous version of OpenWrt. In the current case we need to perform this process for the latest version - Chaos Calmer. So please replace all the URLs that point to tools for Attitude Adjustment to that of Chaos Calmer. The respective URLs for Chaos Calmer are at the end of the blog post.
  3. Compile OpenWrt for TL-MR3020 on your ubuntu laptop:
    1. Download https://downloads.openwrt.org/chaos_calmer/15.05/ar71xx/generic/OpenWrt-ImageBuilder-15.05-ar71xx-generic.Linux-x86_64.tar.bz2 to your 64-bit Ubuntu box
    2. Extract the archive:
      bzcat OpenWrt-ImageBuilder-15.05-ar71xx-generic.Linux-x86_64.tar.bz2  | tar -xvf -
    3. Go into the folder and run:
      make image PROFILE=TLMR3020 PACKAGES="blkid block-mount kmod-fs-ext4 kmod-usb2 kmod-usb-uhci kmod-usb-ohci kmod-usb-storage"
  4. Connect Router to laptop using LAN cable. (If you have a new router, use web console to first flash a generic OpenWRT image of some previous version which is known to work fine. Look here on how to do that. Once that is done, use the OpenWrt webpage to flash the new custom compiled image.) Flash the following image using OpenWrt's web console  ~/OpenWrt-ImageBuilder-15.05-ar71xx-generic.Linux-x86_64/bin/ar71xx/openwrt-15.05-ar71xx-generic-tl-mr3020-v1-squashfs-factory.bin
  5. Disconnect the ethernet cable and connect to your router using a USB cable from your laptop to FT232RL board that you soldered to your TL-MR3020. Now use PuTTY or minicom to gain access to console on your router. Look here on how to configure your router to connect to your home WiFi as a client.
  6. Once the router has successfully connected to your home WiFi, you can either continue using serial console or use PutTTY or SSH over WiFi. Issue commands to the router and mount the external flash drive and shift the rootfs to it:
    1. We need to check the path where MR3020 has mounted the flash drive. Execute the command
      ls /dev/sd*
      to know the path. In my case this was /dev/sdb1. In your case it could be /dev/sda1. Use this path in the following steps wherever required.
    2. The remaining steps required to move the rootfs to the flash drive have been taken from this page here. Mount the USB stick and copy the flash /overlay to the USB stick by executing the following four commands in succession:
      1. mkdir -p /mnt/usb
      2. mount /dev/sdb1 /mnt/usb
      3. tar -C /overlay -cvf - . | tar -C /mnt/usb -xvf -
      4. vi /etc/config/fstab
      5. The last of the above four commands will open a text editor. Press I to insert and modify the file. The should look like this:
        config 'mount'
                option target   /overlay
                option device   /dev/sdb1
                option fstype   ext4
                option options  rw,sync
                option enabled  1
                option enabled_fsck 0
      6. After editing the file press Escape on your keyboard followed by colon (:) and then type "wq!" and press Enter to save the file.
        Reboot the router and check if you have gained the extra space of the USB flash drive using the command df -h
         _______                     ________        __
        |       |.-----.-----.-----.|  |  |  |.----.|  |_
        |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
        |_______||   __|_____|__|__||________||__|  |____|
                  |__| W I R E L E S S   F R E E D O M
        -----------------------------------------------------
        CHAOS CALMER (15.05, r46767)
        -----------------------------------------------------
          * 1 1/2 oz Gin            Shake with a glassful
          * 1/4 oz Triple Sec       of broken ice and pour
          * 3/4 oz Lime Juice       unstrained into a goblet.
          * 1 1/2 oz Orange Juice
          * 1 tsp. Grenadine Syrup
        -----------------------------------------------------
        root@OpenWrt:/# df -h
        Filesystem                Size      Used Available Use% Mounted on
        rootfs                    7.2G     17.0M     
        6.8G   0% /
        /dev/root                 2.3M      2.3M         0 100% /rom
        tmpfs                    14.0M     72.0K     14.0M   1% /tmp
        /dev/sdb1                 7.2G     17.0M      6.8G   0% /overlay
        overlayfs:/overlay        7.2G     17.0M      6.8G   0% /
        tmpfs                   512.0K         0    512.0K   0% /dev
        root@OpenWrt:/#
  7. Next install the packages which we had left out while compiling OpenWrt. Make sure your TL-MR3020 is connected to home WiFi and therefore the internet and issue these commands to it over PuTTY/SSH:

    opkg update
    opkg install luci
    opkg install kmod-usb-serial-ftdi kmod-usb-acm kmod-usb-serial curl coreutils-stty usbutils bash nano grep getopt ip-full 


    This will install LuCI the web interface as well as few other useful utilities for your IoT projects.
  8. Use web browser to visit 192.168.1.1 from your laptop (use a LAN cable connected between your laptop and TL-MR3020 and your laptop's WiFi must be off)
    Configure whatever your TL-MR3020's settings to whatever you want to.

11 comments:

ram kumar said...

Hi, I followed the same step and built and flashed, sda1 is showing when I list but when I try to mount Im getting following error, please help


root@OpenWrt:~# mount /dev/sda1 /mnt/usb
mount: mounting /dev/sda1 on /mnt/usb failed: Invalid argument



ram kumar said...

[update]

Worked after these steps
opkg install kmod-usb-storage block-mount block-hotplug kmod-fs-ext4 kmod-fs-vfat kmod-nls-cp437 kmod-nls-iso8859-1
mkdir -p /mnt/usb
mount -t vfat /dev/sda1 /mnt/usb



Thanks a lot, love your blog and work

Anurag Chugh said...

@ram kumar,

Thanks for the compliments! Alos, many thanks for the update as well, I have been playing around and blogging about OpenWrt for quite sometime now. I always used to wonder if developers/hobbyist found my blog useful. Comments and updates from readers like you help keep me motivated to keep blogging about what I do. Thank you again.

Dr.crazy said...

Very Good.

Anurag Chugh said...

Updates:
TL-MR3020 Hardware page: https://openwrt.org/toh/tp-link/tl-mr3020

Latest OpenWrt at the time of this comment is 18.06.1 but that does not work on TL-MR3020. For TL-MR3020, the latest is 17.01.5

So if you want the compile your own image, get:
http://downloads.openwrt.org/releases/17.01.5/targets/ar71xx/generic/lede-imagebuilder-17.01.5-ar71xx-generic.Linux-x86_64.tar.xz

To extract it, use:
tar -xvf lede-imagebuilder-17.01.5-ar71xx-generic.Linux-x86_64.tar.xz

To build the image, use the command:
make image PROFILE=tl-mr3020-v1 PACKAGES="blkid block-mount kmod-fs-ext4 kmod-usb2 kmod-usb-uhci kmod-usb-ohci kmod-usb-storage"

I knew the profile name has changed because I ran "make info" first

The compiled output was placed at:
lede-imagebuilder-17.01.6-ar71xx-generic.Linux-x86_64/bin/targets/ar71xx/generic/lede-17.01.6-ar71xx-generic-tl-mr3020-v1-squashfs-factory.bin

Anurag Chugh said...

To connect to your home WiFi without having LuCI (i.e. via console prompt over serial) you need to configure 3 files:
/etc/config/network (DHCP client)
/etc/config/wireless (Sta mode, SSID and Password)
/etc/config/firewall (Open incoming port so that you can later SSH into it, remember you can only SSH into it after you set your passwd)

You can edit the files using vi manually or use uci commands

Anurag Chugh said...

root@LEDE:/# cat /etc/config/network

config interface 'loopback'
option ifname 'lo'
option proto 'static'
option ipaddr '127.0.0.1'
option netmask '255.0.0.0'

config globals 'globals'
option ula_prefix 'fd14:9b45:024b::/48'

config interface 'lan'
option type 'bridge'
option ifname 'eth0'
option proto 'static'
option ipaddr '192.168.1.1'
option netmask '255.255.255.0'
option ip6assign '60'

config interface 'wan'
option proto 'dhcp'


-------------------------

root@LEDE:/# uci show network
network.loopback=interface
network.loopback.ifname='lo'
network.loopback.proto='static'
network.loopback.ipaddr='127.0.0.1'
network.loopback.netmask='255.0.0.0'
network.globals=globals
network.globals.ula_prefix='fd14:9b45:024b::/48'
network.lan=interface
network.lan.type='bridge'
network.lan.ifname='eth0'
network.lan.proto='static'
network.lan.ipaddr='192.168.1.1'
network.lan.netmask='255.255.255.0'
network.lan.ip6assign='60'
network.wan=interface
network.wan.proto='dhcp'

Anurag Chugh said...

root@LEDE:/# cat /etc/config/wireless

config wifi-device 'radio0'
option type 'mac80211'
option channel '11'
option hwmode '11g'
option path 'platform/ar933x_wmac'
option htmode 'HT20'
option disabled '0'

config wifi-iface 'default_radio0'
option device 'radio0'
option mode 'sta'
option ssid 'some_ssid'
option key 'some_password'
option encryption 'psk2'
option network 'wan'



-----

root@LEDE:/# uci show wireless
wireless.radio0=wifi-device
wireless.radio0.type='mac80211'
wireless.radio0.channel='11'
wireless.radio0.hwmode='11g'
wireless.radio0.path='platform/ar933x_wmac'
wireless.radio0.htmode='HT20'
wireless.radio0.disabled='0'
wireless.default_radio0=wifi-iface
wireless.default_radio0.device='radio0'
wireless.default_radio0.mode='sta'
wireless.default_radio0.ssid='some_ssid'
wireless.default_radio0.key='some_password'
wireless.default_radio0.encryption='psk2'
wireless.default_radio0.network='wan'

Anurag Chugh said...

root@LEDE:/# cat /etc/config/firewall

config defaults
option syn_flood '1'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'REJECT'

config zone
option name 'lan'
list network 'lan'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'ACCEPT'

config zone
option name 'wan'
list network 'wan'
list network 'wan6'
option output 'ACCEPT'
option masq '1'
option mtu_fix '1'
option input 'ACCEPT'
option forward 'ACCEPT'

........Rest truncated... important part was to change input to ACCEPT for wan zone


root@LEDE:/# uci show firewall.@zone[1]
firewall.cfg06dc81=zone
firewall.cfg06dc81.name='wan'
firewall.cfg06dc81.network='wan' 'wan6'
firewall.cfg06dc81.output='ACCEPT'
firewall.cfg06dc81.masq='1'
firewall.cfg06dc81.mtu_fix='1'
firewall.cfg06dc81.input='ACCEPT'
firewall.cfg06dc81.forward='ACCEPT'

Anurag Chugh said...

root@LEDE:/# df -h
Filesystem Size Used Available Use% Mounted on
/dev/root 2.3M 2.3M 0 100% /rom
tmpfs 13.7M 72.0K 13.7M 1% /tmp
/dev/sda1 7.3G 69.2M 6.8G 1% /overlay
overlayfs:/overlay 7.3G 69.2M 6.8G 1% /
tmpfs 512.0K 0 512.0K 0% /dev

Anurag Chugh said...

root@LEDE:/# opkg list-installed
base-files
bash
bc
blkid
block-mount
busybox
ca-certificates
coreutils
coreutils-stty
curl
dnsmasq
dropbear
firewall
fstools
fwtool
getopt
grep
hostapd-common
ip-full
ip6tables
iptables
iw
jansson
jshn
jsonfilter
kernel
kmod-ath
kmod-ath9k
kmod-ath9k-common
kmod-cfg80211
kmod-crypto-crc32c
kmod-crypto-hash
kmod-fs-ext4
kmod-gpio-button-hotplug
kmod-ip6tables
kmod-ipt-conntrack
kmod-ipt-core
kmod-ipt-nat
kmod-lib-crc-ccitt
kmod-lib-crc16
kmod-mac80211
kmod-nf-conntrack
kmod-nf-conntrack6
kmod-nf-ipt
kmod-nf-ipt6
kmod-nf-nat
kmod-nls-base
kmod-ppp
kmod-pppoe
kmod-pppox
kmod-scsi-core
kmod-slhc
kmod-usb-acm
kmod-usb-core
kmod-usb-ledtrig-usbport
kmod-usb-ohci
kmod-usb-serial
kmod-usb-serial-ch341
kmod-usb-serial-ftdi
kmod-usb-storage
kmod-usb-uhci
kmod-usb2
knot-dig
knot-libdnssec
knot-libknot
lede-keyring
libblkid
libblobmsg-json
libbz2
libc
libcurl
libdb47
libffi
libgcc
libgdbm
libgmp
libgnutls
libip4tc
libip6tc
libiwinfo
libiwinfo-lua
libjson-c
libjson-script
liblua
liblzma
libmbedtls
libmodbus
libncurses
libnettle
libnl-tiny
libopenssl
libpcre
libpthread
libreadline
librt
libsqlite3
libstdcpp
libubox
libubus
libubus-lua
libuci
libuci-lua
libuclient
libusb-1.0
libuuid
libxml2
libxtables
logd
lua
luci
luci-app-firewall
luci-base
luci-lib-ip
luci-lib-jsonc
luci-lib-nixio
luci-mod-admin-full
luci-proto-ipv6
luci-proto-ppp
luci-theme-bootstrap
minicom
mtd
nano
netifd
odhcp6c
odhcpd
opkg
ppp
ppp-mod-pppoe
procd
protobuf
python3
python3-asyncio
python3-base
python3-codecs
python3-ctypes
python3-dbm
python3-decimal
python3-distutils
python3-email
python3-gdbm
python3-light
python3-logging
python3-lzma
python3-multiprocessing
python3-ncurses
python3-openssl
python3-pydoc
python3-sqlite3
python3-unittest
python3-xml
rpcd
swconfig
terminfo
uboot-envtools
ubox
ubus
ubusd
uci
uclient-fetch
uhttpd
uhttpd-mod-ubus
usbutils
usign
wpad-mini
zlib

Post a Comment