Thursday, February 15, 2018

Linux Internals

What happens at boot?

  1. Kernel Boot Process by 0xAX at Github
  2. Analyzing the Linux boot process by Alison Chaiken

What happens when a device is plugged in?

  1. What actually happens when you plug in a USB device? by Ian Wienand

What happens when you execute a userspace programm from the command line?

  1. How to show all shared libraries used by executables in Linux? Stack Overflow
  2. Loading of shared libraries and RAM usage Unix & Linux Stack Exchange

What happens when you compile userspace applications?

  1. How to write shared libraries? by Ulrich Drepper
  2. Static, Shared Dynamic and Loadable Linux Libraries YoLinux.com
  3. Load-time relocation of shared libraries eli.thegreenplace.net
  4. Position Independent Code (PIC) in shared libraries eli.thegreenplace.net
  5. Where the top of the stack is on x86 eli.thegreenplace.net

What happens when you compile the kernel?

  1. Shrinking the kernel with link-time garbage collection LWN.net
  2. Shrinking the kernel with link-time optimization LWN.net
  3. Shrinking the kernel with an axe LWN.net

Thursday, February 1, 2018

Hacking "ESP8266 Black Board T5" Part 1 - Blinky on Keil

A few months ago, I got my hands on ESP8266 Black cloud features Board T5 which has an ESP8266, an 8051 microcontroller and a few sensors. The 8051 microcontroller acts as a master to the ESP8266 and comes loaded with a demo program which is supposed to work with some android app. ESP8266 can be directly programmed using Arduino, so it would be better if I would reprogram the 8051 to act as a slave connected to ESP8266. The ESP8266 will run an Arduino sketch (that I would write) and ask the 8051 for the sensor readings (over UART). To do that I would need to program the 8051 with come C program that I have written.

So the first step was to learn how to compile a simple blinky program for the 8051 and be able to download and run it on it.

Links:

  1. Black Board T5 User guide (in Chinese) explain how to use the android app (Translated version)
  2. Reworking the AI Thinker "Black board T5
  3. Schematic of Black Board T5 (You will need Batang font to view it)
  4. Source code (Keil uVision project) of the original code running inside 8051
  5. Datasheet of STC15L2K32S2 (The 8051 Microcontroller present on Black board T5)
  6. Android app and its source code
  7. Keil C51 Evaluation Kit ver 9.57 - C compiler for 8051
  8. Windows utility (stc-isp-15xx-v6.86I.exe) to program the flash memory of STC15L2K32S2 over its UART. The utility is in Chinese so you have to feel your way around it.

Schematic for Blackboard
STC15L2K32S2 has two UARTs (Serial Ports). One is connected to ESP8266 and the other one is brought out on the 3 pin header. The one brought out on the 3 pin header can be used to reprogram the microcontroller using the STC's ISP utility (In System Programming)

I used a USB-Serial convertor to connect 8051's UART like so:

TxD, RxD, GND from CH340 board connected to Blackboad T5's R,T,G Signals respectively

To run blinky on 8051, I performed the following steps:
  1. Installed Keil C51 Evaluation Kit on my Windows Laptop
  2. Connect the USB-Serial board to Blackboard
  3. Install drivers for CH340 present on my USB-Serial board
  4. Use the STC utility to install headers for STC microcontrollers to Keil. Keil doesn't come with support for STC microcontrollers built-in.
  5. Use the STC utility to generate demo code for controlling PWM on STC15L2K32S2. Instead of copying the C code, click "Save project" to save a Keil C51 project - that way all the settings are exported as well.
  6. Open the project in keil and build the project by pressing the F7 key. Hex file will be generated. The evaluation version is limited to generating 2kB of code, which is fine if you just want to blink LED using the PWM feature of STC15L2K32S2.
  7. Go back to STC utility, select the MCU type as STC15L2K32S2
  8. Select the appropriate COM Port
  9. Load the hex file onto the STC utility (click Open Code File)
  10. While the Black board T5 is connected to your computer via the CH340 board, keep the batteries installed in Black board T5 but keep it powered off
  11. Click on "Download/Program" in STC utility
  12. Turn on the power via the slide switch on the black board and watch the program download.
  13. Once the program is complete, the code will run and LEDs will fade in and out.

STC Utility: Install headers for STC microcontrollers to Keil

STC Utility: Generate demo code

Keil C51 Evaluation kit: Build the program to generate hex file

STC Utility: select the correct microcontroller, com port,
load the hex file and click Download/Program


Incase you aren't able to generate code from the STC utility, here is the link to the code (Keil C51 project): STC15L2K32S2_Hello_LED.rar

Wednesday, January 31, 2018

googletest on Ubuntu

Running gtests on your C++ programs using g++ on Ubuntu:

Installation


Install CMake:
  • sudo apt install cmake
  • Switch to home directory and:
  • git clone --depth 1 https://github.com/google/googletest.git

Build it:
  • cd ~/googletest
  • mkdir build
  • cd build
  • cmake -Dgtest_build_samples=ON -Dgtest_build_tests=ON ~/googletest
  • make
  • make test
  • ./googlemock/gtest/sample1_unittest


Install the googletest and googlemock .a and .h files:
  • sudo cp -r -v ~/googletest/googletest/include/gtest /usr/local/include
  • sudo cp -r -v ~/googletest/googlemock/include/gmock /usr/local/include
  • sudo cp -v ~/googletest/build/googlemock/lib*.a /usr/local/lib
  • sudo cp -v ~/googletest/build/googlemock/gtest/lib*.a /usr/local/lib



TEST() and code in same file - Test Fail

To compile and run:
  • g++ 01sqrt_fail.cpp -lgtest -lpthread -o 01sqrt_fail
  • ./01sqrt_fail
Output:




TEST() and code in same file - Test Pass

To compile and run:
  • g++ 01sqrt_pass.cpp -lgtest -lpthread -o 01sqrt_pass
  • ./01sqrt_pass
Output:




TEST_F() and code in same file - Test Pass

To compile and run:
  • g++ 02bankacc_pass.cpp -lgtest -lpthread -o 02bankacc_pass
  • ./02bankacc_pass
Output:




TEST_P() and code in same file - Test Fail

To compile and run:
  • g++ 03bankacc_fail.cpp -std=c++11 -lgtest -lpthread -o 03bankacc_fail
  • ./03bankacc_fail
Output:




TEST_P() and code in same file - Test Fail and Formatted Output

To compile and run:
  • g++ 03bankacc_format.cpp -std=c++11 -lgtest -lpthread -o 03bankacc_format
  • ./03bankacc_format
Output:

References

  1. Repository and Readme.md:
    https://github.com/google/googletest

  2. Build and install Google Test:
    https://gist.github.com/massenz/41bb2c8375294f4d9927

  3. Primer:
    https://github.com/google/googletest/blob/master/googletest/docs/Primer.md

  4. C++ Unit Testing with Google Test Tutorial (Youtube):
    https://www.youtube.com/watch?v=16FI1-d2P4E

  5. StackOverflow: How to configure and setup google test framework in linux https://stackoverflow.com/questions/19810731/how-to-configure-and-setup-google-test-framework-in-linux

  6. Some more gtest samples:
    https://github.com/google/googletest/blob/master/googletest/docs/Samples.md

Link to google doc containing above blog post

Wednesday, January 10, 2018

MNO LAN: Use same MNOs SIM card in your IoT devices like GL-MiFi

One of the big issues with IoT is somehow enabling two end nodes to communicate with each other in the presence of Network Address Translation (NAT)
Now days most ISPs (and Mobile Network Operators or MNOs) use NAT by default and assign you local IP addresses in 10.*.*.* subnet. You usually have to pay an extra fee to get static global IP addresses assigned to you - this could be a security concern.
If your end nodes run embedded linux, the following solutions can be used:


  1. Use DDNS and UPnP together
    DDNS is not usually free, but the manufacturer of your embedded linux device (routers from goodlife of Vstarcam's IP camera) might provide you this service for free as it uses almost nil resources on the server side. Most ISPs have UPnP support disabled on their gateway. In a home environment, if you have your own static global IP for your home router, you can use UPnP and DDNS to your advantage. An end node will use UPnP to request your WiFi Router to open ad forward certain outward facing ports to it and then. It will then detect the global IP address of its parent's gateway (in your case, your own home router) and register it with its DDNS server. You can then use the URL assigned to the device to access it from anywhere in the world.
  2. Use Virtual Private Networking
    Pay for a virtual private networking service and use OpenVPN clients on your end devices and bring them all on the same LAN
  3. Use yaler.net
    Its a paid reverse SSH relay service. Works well for SSH and HTTPS connections to your device
  4. Use a server with a global IP address
    Rent virtual VM from digital ocean or amazon or google and develop software for your devices to send data to or receive data from. this is what most manufacturers do for their IoT appliances.

There is a 5th option if:

  1. You are working on a DIY project - handful of embedded linux devices with SSH access.
  2. Your devices are located in the same geographical area (i.e. within the same telecom circle)
  3. Your devices have 3G/4G modem
The trick is to stick the same network operators SIM card in all your devices. All these devices will be assigned IP addresses on the same subnet and would be able to ping each other.

The 10.*.*.* only allows around 16 million unique IP addresses and so I have reason to believe that in India, different MNO circles would be on different logical LANs

Here are the results of my tests - I used two GL-MiFi with 4G modems in them with Reliance Jio 4G SIM cards. Both the devices were located within Maharashtra (one in Pune and other in Talegaon)

I opened the outward facing ports on GL-MiFi and deactivated the firewall on them.
I then conducted iperf tests and here is what I got. It seems that we can easily transfer data between the devices at around 1-2 Mbps while on the Jio network.

Results. PDF is here.


TurboPower

We all want our smartphone to charge quickly. And different phone manufacturers have either created their own propriety method of charging (Oppo VOOC or MediaTek PumpExpress+ or Motorola's TurboPower or OnePlus's Dash) or have opted to comply with a industry wide standard like Qualcomm's Quick Charge series of standards).
These technology work my allowing a phone to tell the charger to step up the voltage beyond 5 volts so that the power available to the phone for charging the battery is more than what a good 5 volt / 2 ampere wall/car charger can supply.

I have a Moto G4 plus and I wanted to make sure that I had access to fast charging at home/office, in car or while travelling. So I went ahead and got the following accessories

My Moto G4 plus along with the various charging accessories

Voltmeter

USB Voltmeter Stick

  • This small device is what allowed me to check at what voltage/current the various chargers were charging my phone or regular devices.
  • It has a USB male and female receptacle so that it can sit between the charger and the USB cable.
  • It shows the voltage, current as well total mAh consumed during the past few hours - all these 4 parameters are shown on its OLED display. It ofcourse discounts the current it itself requires.
  • A reset button will allow you to reset the energy and time counter - The energy and time counters are stored in its non volatile memory and their values are preserved across power cycles so you need to reset them manually.
  • ₹777.00 at Amazon.in

Original TurboPower Charger that cam with your mobile phone

Wall Charger
  • Comes in the box when you buy your phone
  • There re two varieties of these chargers - one has a USB female socket in it and comes with a  detachable microUSB cable while others have microUSB cable connected to it permanently. I have both of these but prefer the one with USB female socket
  • When I used the voltmeter to investigate the charger I discovered the following:
    • It charges regular devices at 5.0 volts
    • It charges Moto G4 plus and Moto G5s plus at around 9.0 volts
    • It charges Qualcomm QC 3.0 compatible devices at around 11.5 volts
  • Thankfully, its 2 pin has the correct diameter and distance between them to fit snugly in Indian AC main sockets - unlike those sad chargers which have europlugs which hang loosely when used with Indian sockets.
  • ₹1,364.00 at Amazon.in 
Charging a regular device at 5.0 volts

Charging Moto G4 plus at around 9.0 volts


Charging a QC 3.0 Power Bank at around 11.5 volts

TurboPower Car Charger


Motorola Turbo Power 15W Qualcomm 2.0 Quick Charge Car Charger (Black)
Car Charger
  • This charger is small and compact and is an original Motorola accessory which is bought separately.
  • Supports QC 2.0 devices (Qualcomm QuickCharge)
  • When I used the voltmeter to investigate the charger I discovered the following:
    • It charges regular devices at 5.0 volts
    • It charges Moto G4 plus and Moto G5s plus at around 9.0 volts
    • It charges Qualcomm QC 3.0 compatible devices at around 11.5 volts
  • ₹990.00 at Amazon.in 
Charging a regular device at 5.0 volts
Charging Moto G4 plus at around 9.0 volts
Charging a QC 3.0 Power Bank at around 11.5 volts

LCARE Qualcomm QC 3.0 Quick Charge Power Bank


  • Now that I had charger for fixed sockets (Car and AC Mains) I wanted a portable power bank with fast charging technology as well. Motorola's TurboPower seemed compatible with QC 2.0/3.0 so I tried this particular powerbank and was pleasantly surprised.
  • When I used the voltmeter to investigate the charger I discovered the following:
    • It charges regular devices at 5.0 volts
    • It charges Moto G4 plus and Moto G5s plus at around 9.0 volts
  • The QC 3.0 technology is used in input as well as output sides. i.e. when you try to charge the power bank itself using any of motorola's original accessory - the car charger and the wall charger - you notice that those chargers themselves are able to charge the powerbank at around 11.5 volts - so basically you have a power bank that not only charges other devices quickly, but also itself gets charged quickly.
  • A backlit LCD display shows the charge level remaining in percentage numbers and also shows the charging technology being used
  • One downside - the powerbank cannot charge mobile devices when it is itself being charged.
  • ₹1,755.00 at Amazon.in
Powerbank charging  moto G4 plus at around 9.0 volts

Powerbank charging a regular bluetooth headset at 5.0 volts

Sunday, January 7, 2018

Food Silicone food savers + Glass bottles

Plastics! Cancer! Correlation!
And so we move away from plastic bottles to glass bottles.
But glass bottles pretty but fragile.
So what do we do?

We get food huggers!!!
Food huggers seemed onle somewhat useful to me all this while until now
The silicone rubber food huggers will protect the bottom of the bottles - I was worried that I might end up banging them on the floor when I have a sip in the middle of the night.


Color Glass bottles
 +
Silicone food hugger


 =



Amazon.in links:
  1. Food huggers/savers
  2. Glass bottles
Note:

  1. The food huggers come as a set of 4. Only the two larger ones have diameters large enough for the glass bottles linked to above. The smaller two won't fit.
    So if you get 6 bottles, you will need to order 3 sets of food huggers. Of the 12 pieces of food huggers, you will end up using only 6 of them.
  2. Various sized food huggers seem to be colored differently to match the color of the vegetable/fruit they are supposed to protect. Yellow is for lemons, Purple for onions, Green for capsicum, Red for tomatoes. This is a good thing - food huggers come in various colors and so do the glass bottles. So there is a very high chance that you would end up getting matching cushions for each bottle except for the blue one.
  3. These silicone rubber huggers can also be used for green tea flasks as well.
  4. Also, here are some photos on how these huggers can protect half cut vegetables from drying up when left in the fridge.







Friday, January 5, 2018

Adding new elements to OpenWrt's LuCI on GL-MiFi

If you are an IoT enthusiast who uses devices that run OpenWrt (routers from Goodlife, or MT7688), you would have considered creating a plugin for LuCI so that you can expose the functionality of any sensors/actuators that you may have connected to your OpenWrt box via OpenWrt's builtin web console.

This article on openwrt.org's wiki will tell you exactly how to do that.
As maybe obvious from the page, if you want to create dynamically generated web content, you will need to write lua scripts.

When trying out the steps from the wiki article for my GL-MiFi, I had to modify the file locations for placing various script like so:


  1. new_tab.lua had to be placed in:
    /usr/lib/lua/luci/controller/myapp
  2. cbi_tab.lua had to be placed in:
    /usr/lib/lua/luci/model/cbi/myapp-mymodule
  3. cbi_file can be put in the same place as mentioned:
    /etc/config
  4. view_tab.htm had to be placed in:
    /usr/lib/lua/luci/view/myapp-mymodule
Once I had the placed the files in the right location, I could see them in the web UI:



My next step is to modify the lua scripts to run executables in C to query the various sensors that I have connected to my OpenWrt devices and display the latest values within a tab in WebUI.

I use the device specifiy SDK from OpenWrt to compile my C programs into executables.

Later on I will package my lua scripts and C programs into a simple .ipkg packages that one can install on any OpenWrt device

Saturday, August 26, 2017

Build system comparison for C shared library: Manual vs. Makefile vs. Bazel

Here is a simple code for shared library (.so) and an executable that uses it:

Library Source files (libhello.c and libhello.h):


Source for the executable that uses the above library (main.c):

There are many ways to build and execute these sources. Lets look at three of them. The first one is a collection of manual commands that you need to execute in a particular order. The other two uses some form of build system. Before you proceed, make sure to have the C compiler installed on your Ubuntu PC. To do this, execute the following at the command prompt:

sudo apt-get install build-essential git


Manual build:

(Reference: http://www.cprogramming.com/tutorial/shared-libraries-linux-gcc.html)
Place the three source files in the same directory. Open the command prompt, switch to the same directory and execute the following commands to compile, execute and clean:
  1. Compilation
    1. Compiling the library sources into object files with Position Independent Code:
      gcc -c -Wall -Werror -fpic libhello.c
    2. Creating a shared library from an object file
      gcc -shared -o libhello.so libhello.o
    3. Linking with a shared library
      (prefix "lib" will be automatically added to library name, -L is used to specify search folder)
      gcc -L$(pwd) -Wall -o main.exe main.c -lhello

      if there are any issues, use the command to debug linker: ld -lhello --verbose
  2. Running the executable
    1. Making the library available at runtime
      export LD_LIBRARY_PATH=$(pwd):$LD_LIBRARY_PATH
    2.  Run the binary
      ./main.exe
  3. Cleaning
    1. Execute:
      rm -rf *.o *.so *.exe

Building using Makefile:

Place the three source files in the same directory. Also create and place a new file "Makefile" with the following contents in the same directory:

Open the command prompt, switch to the same directory and execute the following commands to compile, execute and clean:
  1. Compilation
    • make all
  2. Running the executable
    1. Making the library available at runtime
      export LD_LIBRARY_PATH=$(pwd):$LD_LIBRARY_PATH
    2.  Run the binary
      ./main.exe
  3. Cleaning
    • make clean

Building using Bazel:

To use Bazel (The build system from Google), you need to first install it. Do so using following commands:

  1. Install JDK 8
    • sudo apt-get install openjdk-8-jdk
  2. Add Bazel distribution URI as a package source (one time setup):
    • echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list
    • curl https://bazel.build/bazel-release.pub.gpg | sudo apt-key add -
  3. Install and update Bazel
    • sudo apt-get update && sudo apt-get install bazel
Next, create a directory structure like so:

Current folder
├────WORKSPACE (Blank file)

└───main (folder)
     ├───BUILD (Bazel build script)
     ├───libhello.c
     ├───libhello.h
     └───main.c

The content of the BUILD file should be as follows:

Now carry out the following commands to build the file using Bazel from the command prompt after changing to the directory with blank "WORKSPACE" file:
  1. Compilation
    • bazel build //main:all
  2. Running the executable
    • ./bazel-bin/main/main.exe
  3. Cleaning
    • bazel clean
Compiling, running and cleaning using Bazel

Output files created in the working directory by Bazel
By default, (when linkstatic option is not specified in BUILD file), Bazel links libraries to executables statically. To disable that, specify linkstatic = 0

Monday, July 17, 2017

My favourite alias in .bashrc on Ubuntu


Colorizing and handling system/kernel logs
  • alias logread='cat /var/log/syslog | ccze -A'
  • alias logclear='sudo truncate -s 0 /var/log/syslog'
  • alias logsave='cat /var/log/syslog | ccze -h > ~/log.html'
  • alias dmesgread='dmesg | ccze -A'
  • alias dmesgclear='sudo dmesg -c'
  • alias dmesgsave='dmesg | ccze -h > ~/log.html'
You will have to install ccze (sudo apt-get install ccze)


Thursday, July 13, 2017

Replacing batteries in Black and Decker DP240 2.4V Direct-Plug Rechargeable Screwdriver

I have owned a Black and Decker DP240 Cordless Electric Screwdriver for 5 years now. Its battery had started losing its capacity to hold charge, so I looked online to see if it was possible to change its batteries. I couldn't find any blog/DIY guide on which batteries to replace the internal ones with and how to go about pry open the screwdriver and putting it back. So here are some photos outlining how I changed the batteries in mine and turned it into new.

To open the screwdrive, you will need a set of "Torx Pin 6 lobe" screwdriver set. You cant use regular torx screwdriver bits. You need ones with holes in the center of the bits so that the center pin on the screw can fit into those holes. You can buy just a set of just the bits here.

Torx pin 6 lobe Screwdrivers to open the shell

Screwdriver with the shell opened

It seems that the screwdriver comes with two AA size
1.2V NiCd 800 mAh cells installed


I replaced the 800mAh batteries with 2550 mAh NiMH cells
eneloop by Panasonic is the best brand of NiMH cells


Soldering wires onto the cells

And putting the screw driver back together