Charles Zhu's blog

A journey into basic hardware design with an Arduino Uno R3

2021-12-21T00:00:00Z

I recently decided to obtain an Arduino Uno R3, which, if we are to trust its online reputation, is supposed to be a promising platform for prototyping hardware development. The device came in a starter kit box with some other provided components such as resistors, a breadboard, a USB cable, LCD display, wires, LEDs, a tri-state 8-bit shift register, and a variety of sensors. It is fair to say that the kit promised a decent variety of components for a beginner to dive into Arduino hardware experimentation.

Having obtained the hardware, the next step was to set up a development environment. The Arduino website provided two options for my computer: I could use the Arduino IDE, or use the Arduino command-line utility (arduino-cli). In theory, everything that could be done in the official IDE could be done using the command-line utility. I counterintuitively decided to try using the command-line utility first. I was able to download it for macOS using Homebrew (brew install arduino-cli). Then, I decided to use VS Code to install the official Arduino extension made by Microsoft.

The extension did not work with arduino-cli out the box. It did not immediately detect the plugged in device (although it was detected by running arduino-cli board list in the shell). Several settings had to be set to allow the device to work: "arduino.commandPath": "arduino-cli", "arduino.enableUSBDetection": true, "arduino.useArduinoCli": true, "arduino.path": "/opt/homebrew/bin/".

To make a project, the next step was to run arduino-cli new BareMinimum in the shell, where BareMinimum was my project title. This also works if you manually make a folder and then make a file called BareMinimum.ino within it. Within it you can put at a minimum the following code:

void setup() {
  // put your setup code here, to run once:

}

void loop() {
  // put your main code here, to run repeatedly:

}

After saving the code the next step was to ensure the interface with the actual Uno device. First I could run arduino-cli board list to identify the FQBN (fully qualified board name) for my board. The next step was to search for available ‘cores’ that would work with the board. The next step was to run arduino-cli board listall uno. Following that I ran arduino-cli core install arduino:avr. I could now compile by running arduino-cli compile --fqbn arduino:avr:uno BareMinimum and then I could upload with arduino-cli upload -p /dev/location --fqbn arduino:avr:uno BareMinimum where /dev/location/ would be subsituted by the appropriate /dev entry copied from the output of arduino-cli board list.

After doing all this… the code was uploaded! Nothing of significance happened other than the light from the default LED remaining on. However, by creating another project it would be possible to observe a blinking effect.

void setup() {
    pinMode(LED_BUILTIN, OUTPUT);
}

void loop() {
    digitalWrite(LED_BUILTIN, HIGH);
    delay(1000);
    digitalWrite(LED_BUILTIN, LOW);
    delay(1000);
}

Later I was also able to use the VS Code extension and its board configuration settings to upload code from the IDE directly.

As you might notice from the example code, the Arduino language is a customized dialect of C++.

Following another guide it was possible to proceed to the next step. The next step was to try to use this board with some other hardware! I was able to use a basic LED:

void setup(){
    pinMode(7, OUTPUT);
    digitalWrite(7, LOW);
}

void loop(){ 
}

This was wired as follows (refer to the original link for more details), using a 1K Ω resistor:

I also found it helpful to check resistor values using an online resistor calculator. Resistance values in ohms are typically indicated by colored bands which correspond to certain values. It is important to note which side you begin reading the bands from, since you could obtain different values. An online source suggested reading from the side with grouped bands (never from the side with a metallic band). The resistor here plays the role of limiting current (as we know from the elementary formulation of Ohm’s Law, I = V/R) and therefore a higher resistance could be used to limit further the intensity/brightness of the LED. It is important to use resistors in electronics to protect against excessive current from damaging more sensitive components.

Another interesting feature of this simple circuit is to note that the LEDs used (any LEDs in fact) exhibit a characteristic polarity, that is, current can only flow in one direction and thus the LED must be wired accordingly. LEDs, or light-emitting diodes, like other diodes operate due to a p-n junction which is possible due to the use of semiconductors and impurities which ‘bias’ the current to only flow, for the most part, in one direction. As can be noted on the above image, the 5 volts are on the side of the the anode (+) side of the LED, also wired in series with the resistor. The cathode (-) side is connected to pin 7, which has a ‘LOW OUTPUT’ set by our code, which in according to the code documentation actually sets the pin to 0 volts. In theory, our setup would be similar to doing this:

Or, if we were to follow the standard circuitry schematic, it would be similar to this (albeit with different numerical values for voltage and resistance etc):

Nominally/conventionally, ‘current’ flows from positive to negative, that is, from our anode side to our cathode side (in actuality it is more like the electrons ‘flow’ from the negative to the positive side). Our current works out to be a little less than 5V/1000Ω = 5mA (unknown resistance of the LED itself).

It is also important to note the way that breadboards are wired. Supposedly, breadboards are named as such due to the historic usage of actual bread cutting boards for prototyping circuits. The wiring pattern of breadboards follows this pattern, where each slot within a rows is connected to the others in the row (not going through the middle however), as are each slot in a side column:

This was just a very elementary demonstration of how circuitry could be incorporated with hardware and software, and our humble ATMega328P microcontroller is sufficient to work with this circuitry!

Note: the code snippets in this page come from the linked pages, please refer to linked pages for source information and images as well as further details.

Random number generators, deterministic systems and the difficulty of modeling stochasticity

2022-04-13T00:00:00Z

When using the R language I decided to look up the documentation of the Random function which actually seems to be quite interesting. This Random function in the R base library offers several different pseudorandom number generators: the Wichmann-Hill, Marsaglia-Multicarry, Super Duper, Mersenne-Twister, Knuth-TAOCP-2002, Knuth-TAOCP, and L’Ecuyer-CMRG.

Another blog I found has summarized the apparent performance advantages and disadvantages of each of these random number generators (RNGs) so I will link it here. The data in the post is taken from L’Ecuyer and Simard, 2007.

Firstly some overall philosophical notes about the nature of computers as deterministic state machines. It is interesting that such machines require algorithms such as these to even attempt to simulate stochastic behavior! One thinks of human beings as the opposite of computers: it is easier for us to be “stochastic” and it is hard for us to be reliably “deterministic” and implement (for most people) complex algorithms mentally in an efficient manner. In contrast, a computer in the macro sense can only have a certain amount of states, such that we can consider each possible arrangement of bytes at a given moment in CPU registers, in main memory, in storage etc, to be a finite (though large) number of possible states. That is, (classical) computers suffer a discreteness limitation, as their fundamental operation requires this in accordance with their theoretical model of binary logic, as well as the practical implementation of this.

One notes that (pseudo) RNGs tend to utilize certain “seeds” or certain input values to be fed into RNG algorithms to produce a given pseudorandom number. This tends to be something such as the timestamp (which can reliably be said to different between every usage) although this does not prevent certain problems. Some criticisms have been noted for example of the default rand() function in many C implementations such as that in glibc. The implementation in musl is quite simple:

#include <stdlib.h>
#include <stdint.h>

static uint64_t seed;

void srand(unsigned s)
{
    seed = s-1;
}

int rand(void)
{
    seed = 6364136223846793005ULL*seed + 1;
    return seed>>33;
}

The method utlized by musl appears to be a linear congruential generator, whereas the reference implementation is by Knuth. It is noted that LCGs are not sufficiently secure for “cryptographic applications”. The algorithm follows the formula:

$X_{i+1} = (aX_i + c) \mod m, i=0,1,2,...$

Here a is the constant multiplier, c is the increment, m is the modulus, hence in the above musl implementation, the multiplier is 6364136223846793005, the increment is 1, and the modulus is 2⁶⁴.

The glibc random() code is interesting because it provides very extensive comments. The code comments note:

In addition to the standard rand()/srand() like interface, this package also has a special state info interface. The initstate() routine is called with a seed, an array of bytes, and a count of how many bytes are being passed in; this array is then initialized to contain information for random number generation with that much state information. Good sizes for the amount of state information are 32, 64, 128, and 256 bytes. The state can be switched by calling the setstate() function with the same array as was initialized with initstate(). By default, the package runs with 128 bytes of state information and generates far better random numbers than a linear congruential generator. If the amount of state information is less than 32 bytes, a simple linear congruential R.N.G. is used.

What method is utilized instead of a linear congruential RNG?

The random number generation technique is a linear feedback shift register approach, employing trinomials (since there are fewer terms to sum up that way).

The man page states:

The random() function uses a nonlinear additive feedback random number generator employing a default table of size 31 long integers to return successive pseudo-random numbers in the range from 0 to RAND_MAX. The period of this random number generator is very large, approximately 16 * ((2^31) - 1).

In the code we observe the table:

static int32_t randtbl[DEG_3 + 1] =
  {
    TYPE_3,
    -1726662223, 379960547, 1735697613, 1040273694, 1313901226,
    1627687941, -179304937, -2073333483, 1780058412, -1989503057,
    -615974602, 344556628, 939512070, -1249116260, 1507946756,
    -812545463, 154635395, 1388815473, -1926676823, 525320961,
    -1009028674, 968117788, -123449607, 1284210865, 435012392,
    -2017506339, -911064859, -370259173, 1132637927, 1398500161,
    -205601318,
  };

Which such trinomials are utilized? The code details 4 types of RNGs, with TYPE_0 being the simple linear congruential method, the others being trinomials: TYPE_1 $x^7 + x^3 + 1$ , TYPE_2 $x^15 + x + 1$ , and TYPE_3 $x^31 + x^3 + 1$ . These three trinomial equations intersect at the points (0,1), (1,3) and (-1,-1), while additional intersections exist between the pairs at non-integer points:

These trinomials are important enough that lists have been published of them and they have been subject of study.

The method utilized by glibc in random_r() involves the formula:

int32_t val = ((state[0] * 1103515245U) + 12345U) & 0x7fffffff;

In the GNU man page is detailed the differences between random() and random_r():

The random_r() function is like random(3), except that instead of using state information maintained in a global variable, it uses the state information in the argument pointed to by buf, which must have been previously initialized by initstate_r(). The generated random number is returned in the argument result.

A linear feedback shift register utilizes “a linear function of its previous state” as the input bit.

Another set of random number generators in UNIX and UNIX-like systems (such as macOS, BSD, Linux) occurs in the exposed /dev/random and /dev/urandom “files”. Under the UNIX paradigm “everything is a file” so these “files” can be handled by utilities such as cat as if they were files, allowing pipelining to files etc. These produce continuous streams of random bytes which are considered “cryptographically secure”.

Let’s examine the randomness tests contained within the TestU01 suite proposed by L’Ecuyer and Simard, 2007. They first define a “good imitation” of “independent uniform random variables” (colloquially, what we would presumably consider “truly random”):

RNGs for all types of applications are designed so that their output sequence is a good imitation of a sequence of independent uniform random variables, usually over the real interval $(0, 1)$ or over the binary set $\{0, 1\}$ . In the first case, the relevant hypothesis $H_0^A$ to be tested is that the successive output values of the RNG, say $u_0 , u_1 , u_2 , . . .$ , are independent random variables from the uniform distribution over the interval $(0, 1)$ , that is, i.i.d. $U (0, 1)$ . In the second case, $H_0^B$ says that we have a sequence of independent random bits, each taking the value 0 or 1 with equal probabilities independently of the others.

The paper discusses many methods of testing (implemented in their testing suite), a few are given below:

Statistical two-level statistical test
Several authors have advocated and/or applied a two-level (or second-order) procedure for testing RNGs [Fishman 1996; Knuth 1998; L’Ecuyer 1992; Marsaglia 1985]. The idea is to generate N “independent” copies of Y , say $Y_1 , . . . , Y_N$ , by replicating the first-order test N times on disjoint subsequences of the generator’s output. Let F be the theoretical distribution function of Y under $H_0$ . If F is continuous, the transformed observations $U_1 = F (Y_1)$ , . . . , $U_N = F (Y_N )$ are i.i.d. $U (0, 1)$ random variables under $H_0$ . One way of performing the two-level test is to compare the empirical distribution of these $U_j$ ’s to the uniform distribution, via a goodness-of-fit (GOF) test such as those of Kolmogorov- Smirnov, Anderson-Darling, Crámer-von Mises, etc.
[…]
The p-value of the GOF test statistic is computed and $H_0$ is rejected if this p-value is deemed too extreme, as usual.
Testing a stream of real numbers, this involves “measuring global uniformity”, “measuring clustering” (if the numbers tend to be clustered in a certain manner), and “run and gap tests” (counts gap between successive values that land in a certain interval, using the heuristic of Lebesgue measure).
“Testing for n subsequences of length t”

Blackman and Vigna, 2018 discuss some of the weaknesses of “F₂-linear pseudorandom number generators”. They propose a new generator called xoshiro256** which supposedly addresses the failures of previous RNGs including their own Xoshiro128+. This in turn was criticized by O’Neill, 2018, where she compares it to her own PCG random number generator algorithms.

Functional programming, Lisp-like languages, and type theory

2022-07-26T00:00:00Z

I recently came across the concept of functional programming which piqued my interest. The concept of functional programming syntactically follows the convention as established by “Polish notation”; that is, instead of putting operations between variables (eg. $1 + 2$ to add 1 and 2), which reflects our linguistic conventions (especially in subject-verb-object order) and the origins of mathematical expressions from natural language grammars, we instead list the operation first and then the variables or numbers involved, which is more logical for computers.

For example, to add 1 and 2 in Scheme, we write:

(+ 1 2)

which of course evaluates to 3.

I found the SICP book (Structure and Interpretation of Computer Programs), linked below, to give a helpful introduction to Scheme. The book utilizes the simplicity of Scheme to teach programming concepts. Scheme, as a prominent Lisp derivative, in general follows the convention of prefix notation-based syntax. The parentheses seem to indicate a sort of order of operations as you might find in a mathematical expression, and follows a simple syntactical logic, ie. to do $(2 + 3) (5 + 6)$ , we use:

(* (+ 2 3) (+ 5 6))

Functional programming languages correspond also mathematically to the concepts of type theory and formal languages. Agda is a programming language which has been used in mathematical proofs. Functional programming is also widely used in some fields such as finance.

Resources: