Git Hooks with stderr

For example, it’s tricky to validate a commit with pre-commit hook if an app does not return a proper exit code. Here’s a way to validate with the stderr.

  1. Make a temporary file
  2. Redirect stderr to the temp file
  3. Count the length of its content

You can understand it with code snippets below:

PowerShell

1
2
3
4
5
6
7
$errFile = New-TemporaryFile
command > /dev/null 2>$errFile
$err = Get-Content $errFile
If ($err.Length -ne 0) {
Exit 1
}
Exit 0

Unix/Linux shell

1
2
3
4
5
6
7
errFile = $(mktemp)
command > /dev/null 2>$errFile
$l = wc -l $errFile
if [$l -ne 0]; then
exit 1
fi
exit 0

Preventing privacy leak with TuringFonts (substitution cipher)

Download one of web fonts (.woff2) from TuringFonts. Assumed Zebra is used here

Put the font to the web directory and define the below in a stylesheet:

1
2
3
4
5
6
7
@font-face {
font-family: ArialZebra;
src: url("arial_zebra.woff2")
}
.zebra-substitution {
font-family: ArialZebra
}

Assign zebra-substitution class to where it needs

1
<span class="zebra-substitution">[email protected]</span>

See how it looks.

TuringFonts also provides Encoder to get a ciphered text.

Quick start on sending email with Amazon SES on Node.js

Although Amazon provides great documentation, this is what all I needed to send an email with Amazon SES (Simple Email Service) on Node.js.

Get Started with Amazon SES

To start, visit Amazon SES and hit Get started with Amazon SES. Then Amazon asks which region you want to use SES. Note that region for SES doesn’t need to be the same with your other Amazon services.

Verify Your Domain or Email Address

After choosing the region, you will see the page above. I highlighted menus you need. You can use two types of identities, domains and email addresses. If you have your own domain and you want to send an email from the domain, the first one needs to be configured. If you want to send an email from your existing email account such as Gmail and Outlook, the second one needs to be configured. The first one and the second one is not mutually exclusive. You can use both.

Option 1: Verify Your Domain

After entering Domains menu, hit Verify a New Domain.

In the popup, type your domain in interest, check Generate DKIM Settings and hit Verify This Domain.

For a given domain, SES requests you to configure DNS records in order to verify whether the domain is owned by you. SES will try to verify the domain constantly. If verification succeeds, you will see Verification: verified, DKIM Status: verified, Enalbed for: Yes in the dashboard as shows in the above. Then it’s all set.

Option 2: Verify Your Email Address

Verifying an email address is simpler. After entereing Email Addresses menu, hit Verify a New Email Address and type an email address in interest. And then SES will send a verification email to a given email address. Once you visit a link given in the verification email, the email address will be verified. Then it’s all set.

Build Node.js Application Sending Email with SES

Install AWS SDK

1
$ npm install aws-sdk

Configure AWS Credentials

To verify whether your Node.js application has the right permissions to send an email with SES, you need to provide the right credentials to AWS SDK.

Visit IAM Management Console and hit Create New Access Key.

Don’t worry. The keys have been deleted

Remember the created access key and write it down to the following file:

  • Windows: C:\Users\*USERNAME*\.aws\credentials
  • Linux, Unix, or macOS: ~/.aws/credentials
1
2
3
[default]
aws_access_key_id = AKIAJ4LGGMQNZ4IJK5XA
aws_secret_access_key = 00pkJqVkmR70kLpXRoqRaoB6zcpo1OmnVeuKvlyP

Now AWS SDK will read access key from the credential file and prove your application has the right permissions.

Also you can store AWS credentials in JSON file and load it from Node.js.

Note: Access key allows full access to your Amazon AWS. Do not disclose access key publicly. I recommend to create an IAM user and allows only SES

Script Sending Email with SES

The simplest Node.js script sending an email with SES is provided in Amazon Documentation or its GitHub repository, so I’ll skip the entire script here:

Requst a Sending Limit Increase

Initially, your SES operates in sandbox mode andcan send an email only to domains or email addresses which you verified in the above. To be able to send an email to anyone, you need to Request a Sending Limit Increase.

Visit Create Case and create a case to request a sending limit increase.

Positioning system using DecaWave DW1000

DecaWave’s DW1000

DW1000 is a UWB communication and ranging module manufactured by DecaWave.
It provides a fairly good ranging accuracy (< 2 cm). Not only a simple anchor-tag ranging system, let’s make a practical centimeter-level precision indoor positioning system using it

System Design

Overview

A setup is shown in a figure below:

  • A tag
    • A tag is composed of two components—Raspberry Pi and Arduino Pro Mini
    • Raspberry Pi controles a higher level indoor positioning system
      • It triggers Arduino Pro Mini to probe anchors and measure distance between it and each anchor
      • It requests distance measurements on Arduino Pro Mini and locate itself through trilateration
    • Arduino Pro Mini
      • It is required due to similar reasons of anchors
      • It receives commands from Raspberry Pi and executes the corresponding process
        Processes are:
        • Probing anchors nearby and exchanging series of frames with each anchor
          to measure distance
        • Reporting probed anchors and measured distance values
  • Anchors
    • They are deployed in a site of interest and their 2D/3D coordinates are known
    • They are implemented on Arduino Pro Mini for the following reasons:
      • Arduino Pro Mini’s low power consumption
      • Anchor’s fast response requirement (order of milliseconds)
      • Anchor’s simple tasks (receiving and transmitting UWB frames)

An overall sequence diagram is illustrated below. Tag (Arduino) is actually composed of Arduino Pro Mini and DW1000.

State Machine: Tag

A figure below is a state diagram of a tag:

When a Raspberry Pi commands via I2C to scan, it goes into SCAN state and broadcasts a PING message. Going into PONG state immediately, it waits for PONG messages from anchors nearby for a certain duration. I set it to 100 ms.

If it discovers 3 or more anchors during the PONG state, it goes into ROUNDROBIN state. It exchanges 4 messages with each discovered anchor sequentially, i.e. POLL to an anchor, POLL_ACK from an anchor, RANGE to an anchor, and RANGE_REPORT from an anchor. After sending POLL and RANGE message, it waits for a response message for a certain timeout of 10 ms.

After 4 messages are exchanged with each anchor, it calculates distance between an anchor using Tx/Rx timestamp. It adopts two-way ranging algorithm, which is a bit computational intensive, but I think it is not that much intensive. For more information about two-way ranging, please refer the official appliclation note APS013 by DecaWave.

State Machine: Anchor

A state diagram of an anchor is simple:

  • It normally stays at IDLE state
  • If it receives a PING message, it tries to send a PONG message with a random delay between 0 and 90 ms to avoid the collision and goes into PENDING_PONG state
  • If it receives a POLL message and sends a POLL_ACK message, it goes into RANGE state and waits for a RANGE message is received
    • If timeout of 10 ms expires, it returns to IDLE state
    • If it receives a RANGE message, it sends a RANGE_REPORT message to a tag

Implementation

The whole implementation can be found in my GitHub repository. I used arduino-dw1000 for controlling DW1000 using Arduino. I followed wiring between Arduino Pro Mini and DW1000 as defined in examples/Ranging{Anchor,Tag} of the library. Only I2C interface is needed to connect Raspberry Pi and Arduino Pro Mini

Building Thrift on Windows 10

Contrast to Linux and macOSs, many are having difficult times building and using Thrift on Windows 10. Most Thrift guides on the internet miss some parts. Here’s the correct way to build and use Thrift on Windows 10.

Dependencies

  • Boost
  • OpenSSL (for secure socket)
  • libevent (for callback)

Boost

  • Download boost. At this time of writing, the latest version is 1.64.0. I downloaded boost_1_64_0-msvc-14.1-64.exe.
    • You must download a correct binary with consideration of OS addressing mode (32/64 bits) and MSVC version
    • You can check MSVC version in a project property page of a project which will use Thrift
  • Execute the downloaded file to extract files. In my case, C:\boost_1_64_0

  • Open **Native Tools Command Prompt** and execute the following commands

    1
    2
    3
    4
    5
    6
    cd C:\boost_1_64_0
    bootstrap
    b2 --build-type=minimal --stagedir=stage/x64 ^
    threading=multi link=static toolset=msvc-14.0 ^
    variant=release address-model=64 architecture=x86 ^
    -j8

    Options are:

    • stagedir: directory to store built files
    • threading=multi: build thread-safe Boost
    • link=static: build static Boost library
    • toolset=msvc-14.0: Visual Studio MSVC (Platform Toolset) used for a project
    • variant=release: Debug/Release configuration used for a project
    • address-model=64: addressing mode of your OS
    • architecture=x86: your CPU architecture

OpenSSL

  • Download OpenSSL v1.0.2L binary (not light version)
    • v1.1.0 causes an SSLv3_method-related error when building Thrift, at least in my case
    • Be careful of the addressing mode of your OS
  • Install OpenSSL by executing the downloaded file. I installed it under C:\OpenSSL-Win64

libevent

  • Download libevent-2.0.22 (v2.1.8 has an issue)
  • Extract it under C\:libevent-2.0.22-stable, or whereever you want
  • Open **Native Tools Command Prompt** and build it by executing nmake -f Makefile.nmake

Building Thrift

  • Download Thrift 0.9.2 (v0.9.3 and later version have an issue) and extract it to somewhere (C:\thrift-0.9.2, in my case)

  • Open a solution thrift.sln under C:\thrift-0.9.2\lib\cpp. There are two projects: libthrift and libthriftnb

  • Open property pages and adjust Target Platform Version and Platform Toolset as the same as a project which you are working on

  • Set parameters as the following:

    • libthrift

      • C/C++ > General > Additional Include Directories
        • C:\OpenSSL-Win64\include, C:\boost_1_64_0
      • Librarian > All Options
        • Additional Dependencies
          • libeay32.lib, ssleay32.lib, libeay32MT.lib, ssleay32MT.lib
        • Additional Library Directories
          • C:\OpenSSL-Win64\lib, C:\OpenSSL-Win64\lib\VC\static

      Note that you need to add other library files and directories according to static/dynamic link and Debug/Release configuration

    • libthriftnb

      • C/C++ > General > Additional Include Directories
        • C:\libevent-2.0.22-stable\include, C:\libevent-2.0.22-stable\WIN32-Code, C:\libevent-2.0.22-stable, C:\boost_1_64_0
  • Build libthrift and libthriftn sequentially

Testing Thrift

  • Grab the example code and make a project for each server and client

  • Get the Thrift compiler and compile Thrift definition files above

  • Set parameters of each project as the following:

    • C/C++ > General > Additional Include Directories

      • C:\boost_1_64_0, C:\thrift-0.9.2\lib\cpp\src\thrift\windows, C:\thrift-0.9.2\lib\cpp\src
    • Linker > All Options

      • Additional Dependencies

        • libthrift.lib
      • Additional Library Directories

        • C\:thrift-0.9.2\lib\cpp\x64\Release, C:\boost_1_64_0\stage\x64\lib

          Note: you may need to add other directories according to Debug/Release configuration

  • Build and start a server project and then a client project. You can see Thrift working

Reading ranging data from DecaWave EVK1000

DecaWave EVK1000 and TREK1000

EVK1000 and TREK1000 are Two-way Ranging Evaluation Kit for DecaWave’s UWB DW1000 chip. You can see and evaluate a ranging result on a screen of each of them. However, to see a positioning result, you need to connect one of them with a PC and use DecaWave’s dedicated software only on Windows. Here’s how to read data from EVK1000 without the dedicated software

Preparing a Driver

This kit has a COM (serial) port. But this port is wrapped with USB interface and operates with a virtual COM port driver

  • On Windows, a driver for this (STM32 Virtual ComPort in FS Mode v1.4.0) can be downloaded at here. After downloading and extracting the file, run <OS>/dpinst_<arch>.exe where <OS> and <arch> are your OS and architecture, respectively. A related thread can be found here
  • On Linux and macOS, when a board is connected, a driver will be automatically loaded

For more detail, please refer FTDI Chip

Reading and Parsing Ranging Data

Configure anchors and a tag by flipping switches as instruction says, and connect an anchor or a tag with a PC. Since it uses a virtual COM port, serial communication parameters are automatically configured and you don’t need to worry about it

When connection is successful, you can see outputs on a serial port terminal as the following:

1
2
3
4
5
6
7
8
9
10
11
mr 01 000004d1 00000000 00000000 00000000 001e 20 403c403c t7:0
mc 01 0000058c 00000000 00000000 00000000 001f 21 00003fe5 t7:0
mr 01 000004ba 00000000 00000000 00000000 001f 21 403c403c t7:0
mc 01 0000059a 00000000 00000000 00000000 0020 22 000040fd t7:0
mr 01 000004c8 00000000 00000000 00000000 0020 22 403c403c t7:0
mc 01 00000579 00000000 00000000 00000000 0021 23 00004215 t7:0
mr 01 000004a7 00000000 00000000 00000000 0021 23 403c403c t7:0
mc 01 0000059a 00000000 00000000 00000000 0022 24 0000432d t7:0
mr 01 000004c8 00000000 00000000 00000000 0022 24 403c403c t7:0
mc 01 00000574 00000000 00000000 00000000 0023 25 00004445 t7:0
...

Here, important columns are from the 3rd to 6th, 7th, 8th columns

  • The 3rd to 6th columns represent a range measurement from each anchor 0 to 3 in hexadecimal
    • For example, 000004d1 represents 0x4d1 or 1233, which is 1.233 m.
  • The 7th column represents a sequence number
  • The 8th column represents a ranging sequence number

Modifying regulations in Intel Wi-Fi card EEPROM

Regulations in EEPROM

Wi-Fi regulation defines various restirctions such as transmission power, initial radiation, and dynamic frequency switching for Wi-Fi to operate in a certain country. Most OSs including Linux contain regulations in their kernels, and we can bypass them by modifying a regulation file or a kernel.

However, certain Wi-Fi cards such as Intel’s have their own regulations in their EEPROM, which takes precedence to kernel’s regulations. Here’s how to modify regulations in Intel Wi-Fi card’s EEPROM. In this article, Intel Ultimate N WiFi Link 5300 is used

Preparing a Tool

iwleeprom is a software to read and write content in an EEPROM. Clone, build and install it:

1
2
3
4
$ git clone [https://github.com/0x90/iwleeprom](https://github.com/0x90/iwleeprom)
$ cd iwleeprom
$ make
$ [sudo] make install

Interpreting a Regulation Flag

First, let’s have a look at what EEPROM contains. To read regulations in EEPROM, type iwleeprom -s:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
$ iwleeprom -s
Supported devices detected:
[1] 0000:03:00.0 [RW] Ultimate N WiFi Link 5300 (8086:4236, 8086:1011)
Select device [1-1] (or 0 to quit): 1
Using device 0000:03:00.0 [RW] Ultimate N WiFi Link 5300
No file names given or patch option selected!
No EEPROM actions will be performed, just write-enable test
Regulatory data from card EEPROM...
Regulatory base: 0156
Channel 1: 0e6f
Channel 2: 0f6f
...
Channel 64: 0f31
...
Channel 1 (HT40): 0a6f
Channel 2 (HT40): 0f6f
...
Channel 157 (HT40): 0f61

What those 0e6f, 0f31 and other things mean? Those are flags defining regulations. So, which bit stands for what? When you are looking into Linux kernel source code, specifically, iwlwifi driver, you can see flag definition as below:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
/**
* enum iwl_eeprom_channel_flags - channel flags in EEPROM
* @EEPROM_CHANNEL_VALID: channel is usable for this SKU/geo
* @EEPROM_CHANNEL_IBSS: usable as an IBSS channel
* @EEPROM_CHANNEL_ACTIVE: active scanning allowed
* @EEPROM_CHANNEL_RADAR: radar detection required
* @EEPROM_CHANNEL_WIDE: 20 MHz channel okay (?)
* @EEPROM_CHANNEL_DFS: dynamic freq selection candidate
*/
enum iwl_eeprom_channel_flags {
EEPROM_CHANNEL_VALID = BIT(0),
EEPROM_CHANNEL_IBSS = BIT(1),
EEPROM_CHANNEL_ACTIVE = BIT(3),
EEPROM_CHANNEL_RADAR = BIT(4),
EEPROM_CHANNEL_WIDE = BIT(5),
EEPROM_CHANNEL_DFS = BIT(7),
};

For example, channel 1 has flags 0e6f, which is 1110 0110 1111 in binary. Its interpretation is:

1
2
3
4
5
6
7
8
1110 0110 1111
.............1 This channel is valid
............1. Not valid for IBSS
...........1.. (Unknown)
..........1... Active scanning is allowed
........0..... RADAR detection is not required
.......1...... Supports wide band
......1....... Candidate for dynamic frequency selection

Reading EEPROM

To read and dump EEPROM, type iwleeprom -o <outfile>:

1
2
3
4
5
6
7
8
9
10
11
12
$ iwleeprom -b -o eeprom_dump
Supported devices detected:
[1] 0000:03:00.0 [RW] Ultimate N WiFi Link 5300 (8086:4236, 8086:1011)
Select device [1-1] (or 0 to quit): 1
Using device 0000:03:00.0 [RW] Ultimate N WiFi Link 5300
Saving dump with byte order: BIG ENDIAN
0000 [................................................................]
0080 [................................................................]
...
0780 [................................................................]

EEPROM has been dumped to eeprom_dump

We can view the dumped EEPROM with xxd command or hex mode (:%!xxd) in vim, and can find channel flags. They are written in the same order of channels, but their offsets are not in linear relationships. You can manually locate an offset of each channel:

1
2
3
4
5
6
7
8
9
10
11
12
$ vim eeprom_dump
:%!xxd
00000000: 5a40 0050 7000 0410 0030 0000 0280 8086 4236 8086 1011 0d01 [email protected]
...
00000150: ---- ---- ---- ---- ---- ---- ---- --0e 6f0f 6f0f 6f0f 6f0f :.........oM.W...o.o.o.o.
00000168: 6f0f 6f0f 6f0f 6f0f 6f0f 6f0f 6f0f 610f 6100 0000 0000 0000 :o.o.o.o.o.o.o.a.a.......
00000180: 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 :........................
00000198: 0000 000f e100 000f e100 000f e100 000f e10f 310f 310f 310f :..................1.1.1.
000001b0: 3100 000f 310f 310f 310f 310f 310f 310f 310f 310f 310f 310f :1...1.1.1.1.1.1.1.1.1.1.
000001c8: 3100 0000 000f a10f a10f a10f a10f a100 000d 0a6f 0f6f 0f6f :1..................o.o.o
000001e0: 0f6f 0f6f 0f6f 0c6f 0000 0ce1 0fe1 0f31 0d31 0e31 0f31 0f31 :.o.o.o.o.......1.1.1.1.1
000001f8: 0f31 0f31 0f61 0f61 :.1.1.a.a

Modifying Regulations

It is easy. Open the dumped EEPROM in vim and set to hex mode (:%!xxd). And change the content as you want. To save changes, reert hex mode (:%!xxd -r) and save it (:w)

Writing New Regulations in EEPROM

After finishing modification, rewrite EEPROM with iwleeprom -m -c -i <infile>:

1
2
3
4
5
6
7
8
9
10
11
12
$ iwleeprom -m -c -i eeprom_dump
Supported devices detected:
[1] 0000:03:00.0 [RW] Ultimate N WiFi Link 5300 (8086:4236, 8086:1011)
Select device [1-1] (or 0 to quit): 1
Using device 0000:03:00.0 [RW] Ultimate N WiFi Link 5300
Dump file byte order: BIG ENDIAN
0000 [================================================================]
0080 [================================================================]
...
0780 [================================================================]

EEPROM has been written from eeprom_dump

Finally, you can check modification is applied via iw phy command