Installing Electron manually for electron-builder

This article tries to resolve an uncommon issue when you encounter the following error and it cannot be resolved automatically, e.g. behind proxy, etc.:

1
Error: Electron failed to install correctly, please delete node_modules/electron and try installing again

Download the right version (with consideration of package.json) of Electron from https://github.com/electron/electron/releases

Unpack the downloaded Electron and put all files under node_modules/electron/dist.

Create path.txt file under node_modules/electron with the following content:

  • Windows: electron.exe
  • Linux: electron
  • macOS: Electron.app/Contents/MacOS/Electron

Note: Make sure there is no whitespace characters (including a newline character) after the filename. This is a problem that I struggled for minutes and hours.

References

Quick start on ANTLR4 in Rust - part3

This is my note in adopting and learning ANTLR4Rust

Series


In the previous article, we implemented a parser with internal state variables. However, if a grammar is huge, it is practically impossible to manage a huge number of state variables. Alternatively, a visitor-like approach can be used.

In an exit method of the root rule in the Listener, we can grab the current context and its children and call custom parser methods:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
impl Listener {
fn hdr(&self, ctx: &HdrContextAll) -> Row {
let row_ctx = ctx.row().unwrap();
self.row(&row_ctx)
} fn row(&self, ctx: &RowContextAll) -> Row {
let mut row = Row::new();
let field_ctx_list = ctx.field_all();
for (_i, field_ctx) in field_ctx_list.iter().enumerate() {
let field = self.field(&field_ctx);
row.push(field);
}
row
} fn field(&self, ctx: &FieldContextAll) -> String {
ctx.get_text()
}
}
impl CSVListener for Listener {
fn exit_csvFile(&mut self, ctx: &CsvFileContext) {
let hdr_ctx = ctx.hdr().unwrap();
let header = self.hdr(&hdr_ctx);
self.csv.header = header;
let row_ctx_list = ctx.row_all();
for (_i, row_ctx) in row_ctx_list.iter().enumerate() {
let row = self.row(&row_ctx);
self.csv.rows.push(row);
}
}
}

The minimal working example can be found here.

Quick start on ANTLR4 in Rust - part2

This is my note in adopting and learning ANTLR4Rust

Series


In the previous article, we built the minimal working example walking a parse tree using a listener. In this article, we are going to store a parsed CSV structure into a variable to use in the future.

Define a CSV structure:

1
2
3
4
5
type Row = Vec<String>;#[derive(Debug)]
struct CSV {
header: Row,
rows: Vec<Row>,
}

Add fields to a listener:

1
2
3
4
5
struct Listener {
csv: Box<CSV>,
add_to_header: bool,
row_to_add: Vec<String>,
}

csv will be a resultant CSV structure. add_to_header and row_to_add are internal state variables to generate a CSV structure.

Implement a listener:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
impl CSVListener for Listener {
fn enter_csvFile(&mut self, _ctx: &CsvFileContext) {}
fn exit_csvFile(&mut self, _ctx: &CsvFileContext) {}
fn enter_hdr(&mut self, _ctx: &HdrContext) {
self.add_to_header = true;
}
fn exit_hdr(&mut self, _ctx: &HdrContext) {
self.csv.header = self.row_to_add.to_vec();
self.row_to_add.clear();
self.add_to_header = false;
}
fn enter_row(&mut self, _ctx: &RowContext) {}
fn exit_row(&mut self, _ctx: &RowContext) {
if self.add_to_header {
return;
}
self.csv.rows.push(self.row_to_add.to_vec());
self.row_to_add.clear();
}
fn enter_field(&mut self, _ctx: &FieldContext) {}
fn exit_field(&mut self, _ctx: &FieldContext) {
self.row_to_add.push(_ctx.get_text());
}
}

Feed an input to a parse and extract the result:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
fn main() {
let input = String::from(
"This, is, a, header
This, is, a, row
",
);
let lexer = CSVLexer::new(Box::new(InputStream::new(input)));
let token_source = CommonTokenStream::new(lexer);
let mut parser = CSVParser::new(Box::new(token_source));
let listener_id = parser.add_parse_listener(Box::new(Listener {
csv: Box::new(CSV {
header: Row::new(),
rows: Vec::new(),
}),
add_to_header: false,
row_to_add: Row::new(),
}));
let result = parser.csvFile();
assert!(result.is_ok());
let listener = parser.remove_parse_listener(listener_id);
let csv = listener.csv;
println!("{:#?}", csv);
}

A tricky part is that we have to extract csv from listener. And to this end, we have to extract listener with listener_id.

The minimal working example can be found here.

Quick start on ANTLR4 in Rust - part1

This is my note in adopting and learning ANTLR4Rust

Series


Install nightly version of Rust (and make it default if you want for convenience).

1
2
$ rustup toolchain install nightly
$ rustup default nightly

Get ANTLR4 runtime for Rust from here:

Prepare a grammar. I will use an example grammar: https://raw.githubusercontent.com/antlr/grammars-v4/master/csv/CSV.g4

Generate a parser:

1
$ java -jar <ANTLR4 runtime path> -Dlanguage=Rust CSV.g4

You will get csvlexer.rs, csvlistener.rs , csvparser.rs . Place them into your project src directory.

Add dependencies in Cargo.toml:

1
2
3
[dependencies]
lazy_static = "1.4"
antlr-rust = "0.1"

Add a feature and import lazy_static macros to the root module:

1
2
3
#![feature(try_blocks)]
#[macro_use]
extern crates lazy_static;

Import common and essential things:

1
2
3
4
use antlr_rust::common_token_stream::CommonTokenStream;
use antlr_rust::input_stream::InputStream;
use antlr_rust::parser_rule_context::ParserRuleContext;
use antlr_rust::tree::{ErrorNode, ParseTreeListener, TerminalNode};

Import grammar-specific things:

1
2
3
4
5
6
mod csvlexer;
mod csvlistener;
mod csvparser;
use csvlexer::CSVLexer;
use csvlistener::CSVListener;
use csvparser::*;

Implement ParseTreeListener, a supertraint of CSVListener:

1
2
3
4
5
6
struct Listener;impl ParseTreeListener for Listener {
fn visit_terminal(&mut self, node: &TerminalNode) {}
fn visit_error_node(&mut self, node: &ErrorNode) {}
fn enter_every_rule(&mut self, ctx: &dyn ParserRuleContext) {}
fn exit_every_rule(&mut self, ctx: &dyn ParserRuleContext) {}
}

Implement CSVListener:

1
2
3
4
5
6
7
8
9
10
impl CSVListener for Listener {
fn enter_csvFile(&mut self, _ctx: &CsvFileContext) {}
fn exit_csvFile(&mut self, _ctx: &CsvFileContext) {}
fn enter_hdr(&mut self, _ctx: &HdrContext) {}
fn exit_hdr(&mut self, _ctx: &HdrContext) {}
fn enter_row(&mut self, _ctx: &RowContext) {}
fn exit_row(&mut self, _ctx: &RowContext) {}
fn enter_field(&mut self, _ctx: &FieldContext) {}
fn exit_field(&mut self, _ctx: &FieldContext) {}
}

Read and parse an input. Note that csvFile in the last line is a rule name in CSV.g4:

1
2
3
4
5
6
7
8
9
10
11
12
fn main() {
let input = String::from(
"This, is, a, header
This, is, a, row
");
let lexer = CSVLexer::new(Box::new(InputStream::new(input)));
let token_source = CommonTokenStream::new(lexer);
let mut parser = CSVParser::new(Box::new(token_source));
parser.add_parse_listener(Box::new(Listener {}));
let result = parser.csvFile();
assert!(result.is_ok());
}

The minimal working example can be found here.

Trilateration using the Levenberg-Marquardt method

True range multilateration is a method to determine the location of a movable vehicle or stationary point in space using multiple ranges (distances) between the vehicle/point and multiple spatially-separated known locations (often termed ‘stations’). — True range multilateration, Wikipedia

We can define a trilateration as an optimization problem to minimize a cost function \(S\) for a given estimate \(\beta\), a tuple of \((x, y, z)\)

$$\begin{aligned}
S(\beta)&=\sum_{i=1}^{N}f_i(\beta)\\
&=\sum_{i=1}^{N}\left[r_i-\sqrt{(x-X_i)^2+(y-Y_i)^2+(z-Z_i)^2}\right]^2
\end{aligned}$$

where \(r_i\) is a measured distnace between an Anchor (i), whose coordinates are \((X_i, Y_i, Z_i)\).

Levenberg-Marquardt method is defined by:

$$\beta_{k+1}=\beta_k-\left(J^TJ+\mu_kdiag\left(J^TJ\right)\right)^{-1}J^Tf(\beta_k)$$

where \(J\) is the Jacobian matrix and \(f\) is a column vector composed of \(f_i\):

$$\begin{aligned}
J&=\begin{bmatrix}
\frac{\partial f_1}{\partial x} & \frac{\partial f_1}{\partial y} & \frac{\partial f_1}{\partial z} \\
\frac{\partial f_2}{\partial x} & \frac{\partial f_2}{\partial y} & \frac{\partial f_2}{\partial z} \\
\vdots & \vdots & \vdots \\
\frac{\partial f_N}{\partial x} & \frac{\partial f_N}{\partial y} & \frac{\partial f_N}{\partial z}
\end{bmatrix}\\
&=\begin{bmatrix}
F_1(x-X_1) & F_1(y-Y_1) & F_1(z-Z_1) \\
F_2(x-X_2) & F_2(y-Y_2) & F_2(z-Z_2) \\
\vdots & \vdots & \vdots \\
F_N(x-X_N) & F_N(y-Y_N) & F_N(z-Z_N)
\end{bmatrix}
\end{aligned}$$

where \(F_i\) is a shorthand of each derivative:

$$F_i=\frac{r_i}{\sqrt{(x-X_i)^2+(y-Y_i)^2+(z-Z_i)^2}}$$

Now, we can estimate \(\beta\) by iterating the equation until it converges.

Git Hooks with stderr

For example, it’s tricky to validate a commit with pre-commit hook if an app does not return a proper exit code. Here’s a way to validate with the stderr.

  1. Make a temporary file
  2. Redirect stderr to the temp file
  3. Count the length of its content

You can understand it with code snippets below:

PowerShell

1
2
3
4
5
6
7
$errFile = New-TemporaryFile
command > /dev/null 2>$errFile
$err = Get-Content $errFile
If ($err.Length -ne 0) {
Exit 1
}
Exit 0

Unix/Linux shell

1
2
3
4
5
6
7
errFile = $(mktemp)
command > /dev/null 2>$errFile
$l = wc -l $errFile
if [$l -ne 0]; then
exit 1
fi
exit 0

Preventing privacy leak with TuringFonts (substitution cipher)

Download one of web fonts (.woff2) from TuringFonts. Assumed Zebra is used here

Put the font to the web directory and define the below in a stylesheet:

1
2
3
4
5
6
7
@font-face {
font-family: ArialZebra;
src: url("arial_zebra.woff2")
}
.zebra-substitution {
font-family: ArialZebra
}

Assign zebra-substitution class to where it needs

1
<span class="zebra-substitution">[email protected]</span>

See how it looks.

TuringFonts also provides Encoder to get a ciphered text.

Quick start on sending email with Amazon SES on Node.js

Although Amazon provides great documentation, this is what all I needed to send an email with Amazon SES (Simple Email Service) on Node.js.

Get Started with Amazon SES

To start, visit Amazon SES and hit Get started with Amazon SES. Then Amazon asks which region you want to use SES. Note that region for SES doesn’t need to be the same with your other Amazon services.

Verify Your Domain or Email Address

After choosing the region, you will see the page above. I highlighted menus you need. You can use two types of identities, domains and email addresses. If you have your own domain and you want to send an email from the domain, the first one needs to be configured. If you want to send an email from your existing email account such as Gmail and Outlook, the second one needs to be configured. The first one and the second one is not mutually exclusive. You can use both.

Option 1: Verify Your Domain

After entering Domains menu, hit Verify a New Domain.

In the popup, type your domain in interest, check Generate DKIM Settings and hit Verify This Domain.

For a given domain, SES requests you to configure DNS records in order to verify whether the domain is owned by you. SES will try to verify the domain constantly. If verification succeeds, you will see Verification: verified, DKIM Status: verified, Enalbed for: Yes in the dashboard as shows in the above. Then it’s all set.

Option 2: Verify Your Email Address

Verifying an email address is simpler. After entereing Email Addresses menu, hit Verify a New Email Address and type an email address in interest. And then SES will send a verification email to a given email address. Once you visit a link given in the verification email, the email address will be verified. Then it’s all set.

Build Node.js Application Sending Email with SES

Install AWS SDK

1
$ npm install aws-sdk

Configure AWS Credentials

To verify whether your Node.js application has the right permissions to send an email with SES, you need to provide the right credentials to AWS SDK.

Visit IAM Management Console and hit Create New Access Key.

Don’t worry. The keys have been deleted

Remember the created access key and write it down to the following file:

  • Windows: C:\Users\*USERNAME*\.aws\credentials
  • Linux, Unix, or macOS: ~/.aws/credentials
1
2
3
[default]
aws_access_key_id = AKIAJ4LGGMQNZ4IJK5XA
aws_secret_access_key = 00pkJqVkmR70kLpXRoqRaoB6zcpo1OmnVeuKvlyP

Now AWS SDK will read access key from the credential file and prove your application has the right permissions.

Also you can store AWS credentials in JSON file and load it from Node.js.

Note: Access key allows full access to your Amazon AWS. Do not disclose access key publicly. I recommend to create an IAM user and allows only SES

Script Sending Email with SES

The simplest Node.js script sending an email with SES is provided in Amazon Documentation or its GitHub repository, so I’ll skip the entire script here:

Requst a Sending Limit Increase

Initially, your SES operates in sandbox mode andcan send an email only to domains or email addresses which you verified in the above. To be able to send an email to anyone, you need to Request a Sending Limit Increase.

Visit Create Case and create a case to request a sending limit increase.

Positioning system using DecaWave DW1000

DecaWave’s DW1000

DW1000 is a UWB communication and ranging module manufactured by DecaWave.
It provides a fairly good ranging accuracy (< 2 cm). Not only a simple anchor-tag ranging system, let’s make a practical centimeter-level precision indoor positioning system using it

System Design

Overview

A setup is shown in a figure below:

  • A tag
    • A tag is composed of two components—Raspberry Pi and Arduino Pro Mini
    • Raspberry Pi controles a higher level indoor positioning system
      • It triggers Arduino Pro Mini to probe anchors and measure distance between it and each anchor
      • It requests distance measurements on Arduino Pro Mini and locate itself through trilateration
    • Arduino Pro Mini
      • It is required due to similar reasons of anchors
      • It receives commands from Raspberry Pi and executes the corresponding process
        Processes are:
        • Probing anchors nearby and exchanging series of frames with each anchor
          to measure distance
        • Reporting probed anchors and measured distance values
  • Anchors
    • They are deployed in a site of interest and their 2D/3D coordinates are known
    • They are implemented on Arduino Pro Mini for the following reasons:
      • Arduino Pro Mini’s low power consumption
      • Anchor’s fast response requirement (order of milliseconds)
      • Anchor’s simple tasks (receiving and transmitting UWB frames)

An overall sequence diagram is illustrated below. Tag (Arduino) is actually composed of Arduino Pro Mini and DW1000.

State Machine: Tag

A figure below is a state diagram of a tag:

When a Raspberry Pi commands via I2C to scan, it goes into SCAN state and broadcasts a PING message. Going into PONG state immediately, it waits for PONG messages from anchors nearby for a certain duration. I set it to 100 ms.

If it discovers 3 or more anchors during the PONG state, it goes into ROUNDROBIN state. It exchanges 4 messages with each discovered anchor sequentially, i.e. POLL to an anchor, POLL_ACK from an anchor, RANGE to an anchor, and RANGE_REPORT from an anchor. After sending POLL and RANGE message, it waits for a response message for a certain timeout of 10 ms.

After 4 messages are exchanged with each anchor, it calculates distance between an anchor using Tx/Rx timestamp. It adopts two-way ranging algorithm, which is a bit computational intensive, but I think it is not that much intensive. For more information about two-way ranging, please refer the official appliclation note APS013 by DecaWave.

State Machine: Anchor

A state diagram of an anchor is simple:

  • It normally stays at IDLE state
  • If it receives a PING message, it tries to send a PONG message with a random delay between 0 and 90 ms to avoid the collision and goes into PENDING_PONG state
  • If it receives a POLL message and sends a POLL_ACK message, it goes into RANGE state and waits for a RANGE message is received
    • If timeout of 10 ms expires, it returns to IDLE state
    • If it receives a RANGE message, it sends a RANGE_REPORT message to a tag

Implementation

The whole implementation can be found in my GitHub repository. I used arduino-dw1000 for controlling DW1000 using Arduino. I followed wiring between Arduino Pro Mini and DW1000 as defined in examples/Ranging{Anchor,Tag} of the library. Only I2C interface is needed to connect Raspberry Pi and Arduino Pro Mini

Building Thrift on Windows 10

Contrast to Linux and macOSs, many are having difficult times building and using Thrift on Windows 10. Most Thrift guides on the internet miss some parts. Here’s the correct way to build and use Thrift on Windows 10.

Dependencies

  • Boost
  • OpenSSL (for secure socket)
  • libevent (for callback)

Boost

  • Download boost. At this time of writing, the latest version is 1.64.0. I downloaded boost_1_64_0-msvc-14.1-64.exe.
    • You must download a correct binary with consideration of OS addressing mode (32/64 bits) and MSVC version
    • You can check MSVC version in a project property page of a project which will use Thrift
  • Execute the downloaded file to extract files. In my case, C:\boost_1_64_0

  • Open **Native Tools Command Prompt** and execute the following commands

    1
    2
    3
    4
    5
    6
    cd C:\boost_1_64_0
    bootstrap
    b2 --build-type=minimal --stagedir=stage/x64 ^
    threading=multi link=static toolset=msvc-14.0 ^
    variant=release address-model=64 architecture=x86 ^
    -j8

    Options are:

    • stagedir: directory to store built files
    • threading=multi: build thread-safe Boost
    • link=static: build static Boost library
    • toolset=msvc-14.0: Visual Studio MSVC (Platform Toolset) used for a project
    • variant=release: Debug/Release configuration used for a project
    • address-model=64: addressing mode of your OS
    • architecture=x86: your CPU architecture

OpenSSL

  • Download OpenSSL v1.0.2L binary (not light version)
    • v1.1.0 causes an SSLv3_method-related error when building Thrift, at least in my case
    • Be careful of the addressing mode of your OS
  • Install OpenSSL by executing the downloaded file. I installed it under C:\OpenSSL-Win64

libevent

  • Download libevent-2.0.22 (v2.1.8 has an issue)
  • Extract it under C\:libevent-2.0.22-stable, or whereever you want
  • Open **Native Tools Command Prompt** and build it by executing nmake -f Makefile.nmake

Building Thrift

  • Download Thrift 0.9.2 (v0.9.3 and later version have an issue) and extract it to somewhere (C:\thrift-0.9.2, in my case)

  • Open a solution thrift.sln under C:\thrift-0.9.2\lib\cpp. There are two projects: libthrift and libthriftnb

  • Open property pages and adjust Target Platform Version and Platform Toolset as the same as a project which you are working on

  • Set parameters as the following:

    • libthrift

      • C/C++ > General > Additional Include Directories
        • C:\OpenSSL-Win64\include, C:\boost_1_64_0
      • Librarian > All Options
        • Additional Dependencies
          • libeay32.lib, ssleay32.lib, libeay32MT.lib, ssleay32MT.lib
        • Additional Library Directories
          • C:\OpenSSL-Win64\lib, C:\OpenSSL-Win64\lib\VC\static

      Note that you need to add other library files and directories according to static/dynamic link and Debug/Release configuration

    • libthriftnb

      • C/C++ > General > Additional Include Directories
        • C:\libevent-2.0.22-stable\include, C:\libevent-2.0.22-stable\WIN32-Code, C:\libevent-2.0.22-stable, C:\boost_1_64_0
  • Build libthrift and libthriftn sequentially

Testing Thrift

  • Grab the example code and make a project for each server and client

  • Get the Thrift compiler and compile Thrift definition files above

  • Set parameters of each project as the following:

    • C/C++ > General > Additional Include Directories

      • C:\boost_1_64_0, C:\thrift-0.9.2\lib\cpp\src\thrift\windows, C:\thrift-0.9.2\lib\cpp\src
    • Linker > All Options

      • Additional Dependencies

        • libthrift.lib
      • Additional Library Directories

        • C\:thrift-0.9.2\lib\cpp\x64\Release, C:\boost_1_64_0\stage\x64\lib

          Note: you may need to add other directories according to Debug/Release configuration

  • Build and start a server project and then a client project. You can see Thrift working