Favourite Footguns Pt 1: Leaking references


One annoying design smell I have seen come up a couple of times in different contexts throughout my career is Software Developers leaking references.
This is when an input to an API can change outside its scope and update the state within the API. An example is this wrapper around
a typical data structure. This snippet of code is more or less how Object-Oriented Programming is taught and my IDE pretty much wrote this 
for me. The below examples are programmed with Java 17, but the real-world examples I have seen were not Java bound.

```java
record ListContainer<T>(List<T> list) {

    public Integer size() {
        return list.size();
    }

    public List<T> getList() {
        return list;
    }
}
```

When this API is created it is possible to update the seed array and have that reflected within the ListContainer API. What is worse
is that you can return a new reference of the list to the caller that can in turn make changes upstream. 

```java
public class Application {

    public static void main(String[] args) {

        var seedArray = new ArrayList<>(Arrays.asList(1, 2, 3, 4, 5));
        var listContainer = new ListContainer<>(seedArray);
        System.out.println(listContainer.size()); // ==> 5 👍

        seedArray.add(6);
        System.out.println(listContainer.size()); // ==> 6 👎

        seedArray.add(7);
        seedArray.add(8);
        System.out.println(listContainer.size()); // ==> 8 😅

        var internalList = listContainer.getList();
        internalList.add(9);
        System.out.println(listContainer.size()); // ==> 9 😱
    }
}
```

Although it's not always the case, programmers are expected to become more and more polyglot - _using the right tools for the job_.
This means that you could find developers who describe themselves non-specialists in your language of choice working on **your code**.
Providing idiomatic design to your APIs so you don't hand off a footgun to future consumers is to make fuller use of the type
system and follow function programming principles such as _immutability first_.

```java
record ListContainer<T>(List<T> list) {

    ListContainer(List<T> list) {
        this.list = new ArrayList<>(list);
    }

    public Integer size() {
        return list.size();
    }

    public List<T> getList() {
        return Collections.unmodifiableList(list);
    }
}
```

In the above refactor, a copy of the list is made in the constructor, abandoning the reference so it can be changed elsewhere
without downstream impact. Another unfortunate case with the first edition of the ListContainer API is that it can
transitively leak and cause even more chaos in different parts of the system. By returning an Unmodifiable List 
you are returning an immutable view of that collection, requiring a caller to recreate a list if they choose to update it. 

```java
public class Application {

    public static void main(String[] args) {

        var seedArray = new ArrayList<>(Arrays.asList(1, 2, 3, 4, 5));
        var listContainer = new ListContainer<>(seedArray);
        System.out.println(listContainer.size()); // ==> 5 👍

        seedArray.add(6);
        System.out.println(listContainer.size()); // ==> 5 👍

        seedArray.add(7);
        seedArray.add(8);
        System.out.println(listContainer.size()); // ==> 5 👍

        var internalList = new ArrayList<>(listContainer.getList());
        internalList.add(9);
        System.out.println(listContainer.size()); // ==> 5 👍
    }
}
```


Finding the right words


This is _Take 3_ of starting a blog. I have stories to tell for those who care to read, although finding the right words at the right time is never that easy. On reflection, there isn't any particular blogs I regularly follow. What I have come to realize is the best Software Engineering blogs are not written for other people, but in fact, the author(s) themselves as a journal of their experiences. The rest is just noise. The posts that follow are a journal of what I find interesting, challenging, and fun.

Feel free to reach out or connect with me if you wish to discuss my musings.


Reversing a string like your job depends on it


A team I used to work on used to give what is generally considered an elementary problem as the final coding stage - reversing a string. The argument by the person who proposed this task was that it weeds out those who talk a good game but can't rise to the challenge of simple programming tasks. While there is a debate to be had here about its effectiveness, it did seem to produce interesting results. The team programmed exclusively on the JVM / CLR so using dynamic runtimes like Python, Ruby or JavaScript were not allowed. Having a proficient understanding of the underlying runtime was considered somewhat important.

What I liked about this challenge was the conversation that it started - leading to all sorts of interesting discussions about runtime complexity, text encodings, test-driven design, concurrency concerns when multiple threads were reversing, and generally how computer code is run on a Virtual Machine / CPU. But this leads to an interesting question - what is the best way to reverse a string? In this post I am using Kotlin 1.7.21 with no build tooling, running on JDK 17.

## The obvious

```kotlin
fun reverseString(str: String): String {
    return str.reversed()
}
```

Github Copilot's only recommendation for reversing a string with the `reversed()` function. This is a Kotlin nicety that chains Java's `StringBuilder.reverse().toString()` methods under the hood as part of its standard library text utilities `kotlin.text`. To me, this is the most obvious thing you could do. While this would likely be the sane thing to do in production (if you would ever need to reverse strings in production), it's probably not the answer an interviewer is looking for, as they would likely want you to implment the `reverse()` method yourself.

## My take

I remember my submission - [without the tests] it looked something like this: 

```kotlin
fun reverseString(str: String): String {
    val sb = StringBuilder(str.length)
    for (l in str)
        sb.insert(0, l) // insert at front of buffer
 
 	return sb.toString()
}

fun main() {
    val test1 = reverseString("Howdy, Cowboy!")
    println(test1) 
    // ==> !yobwoC ,ydwoH
}
```

Here, I create a `StringBuilder` object, and instanciate it with the length of the input string. This is so the byte buffer used to store the character sequence doesn't have to dynamically resize after each insert. Also, if there is no provided value in the constructor it will default to a size of 16 characters I then loop over the input string one character at a time, inserting it at the front of the StringBuilders buffer. Once the buffer is filled I then return a string representation of the `StringBuilder` back to the caller. Let's take a closer look at the bytecode...

```kotlin
// Decompiled for Java 17 LTS

public final static reverseString(Ljava/lang/String;)Ljava/lang/String;
  @Lorg/jetbrains/annotations/NotNull;() // invisible
    // annotable parameter count: 1 (visible)
    // annotable parameter count: 1 (invisible)
    @Lorg/jetbrains/annotations/NotNull;() // invisible, parameter 0
   L0
    ALOAD 0
    LDC "str"
    INVOKESTATIC kotlin/jvm/internal/Intrinsics.checkNotNullParameter (Ljava/lang/Object;Ljava/lang/String;)V
   L1
    LINENUMBER 4 L1
    NEW java/lang/StringBuilder
    DUP
    ALOAD 0
    INVOKEVIRTUAL java/lang/String.length ()I
    INVOKESPECIAL java/lang/StringBuilder.<init> (I)V
    ASTORE 1
   L2
    LINENUMBER 5 L2
    ICONST_0
    ISTORE 2
    ALOAD 0
    INVOKEVIRTUAL java/lang/String.length ()I
    ISTORE 3
   L3
    ILOAD 2
    ILOAD 3
    IF_ICMPGE L4
   L5
    ALOAD 0
    ILOAD 2
    INVOKEVIRTUAL java/lang/String.charAt (I)C
    ISTORE 4
   L6
    ALOAD 1
    ICONST_0
    ILOAD 4
    INVOKEVIRTUAL java/lang/StringBuilder.insert (IC)Ljava/lang/StringBuilder;
    POP
   L7
    IINC 2 1
    GOTO L3
   L4
    LINENUMBER 7 L4
    ALOAD 1
    INVOKEVIRTUAL java/lang/StringBuilder.toString ()Ljava/lang/String;
    DUP
    LDC "sb.toString()"
    INVOKESTATIC kotlin/jvm/internal/Intrinsics.checkNotNullExpressionValue (Ljava/lang/Object;Ljava/lang/String;)V
    ARETURN
   L8
    LOCALVARIABLE l C L6 L7 4
    LOCALVARIABLE sb Ljava/lang/StringBuilder; L2 L8 1
    LOCALVARIABLE str Ljava/lang/String; L0 L8 0
    MAXSTACK = 3
    MAXLOCALS = 5
```

Dissembling the method doesn't tell me anything I don't know about how I have written it, but I suspect there might be something more going on behind the scenes of the `StringBuilder` class, particularly the `insert()` method. A suspicion that I didn't have of at the time of writing is what happens when you _insert at the front_ - I can't imagine shifting everything right is a free lunch.

Reading through the OpenJDK implementation, this is what we are calling

```java
// See above Bytecode
// INVOKEVIRTUAL java/lang/StringBuilder.insert (IC)Ljava/lang/StringBuilder;

public AbstractStringBuilder insert(int offset, char[] str) {
    checkOffset(offset, count);
    int len = str.length;
    ensureCapacityInternal(count + len);
    shift(offset, len);
    count += len;
    putCharsAt(offset, str, 0, len);
    return this;
}
```

The first method `checkOffset` checks to see if the caller is trying to insert outside the bounds of the buffer - if it is trying to do that it will throw a `StringIndexOutOfBoundsException`. It's a handy safeguard, but it's not too interesting in this context.

`ensureCapacityInternal` is rather interesting code however - it will resize the buffer if there is not enough capacity for the insert. 

```java
// AbstractStringBuilder.ensureCapacityInternal

private void ensureCapacityInternal(int minimumCapacity) {
    // overflow-conscious code
    int oldCapacity = value.length >> coder;
    if (minimumCapacity - oldCapacity > 0) {
        value = Arrays.copyOf(value,
                newCapacity(minimumCapacity) << coder);
    }
}
```

`shift` is where my suspicion is address.

```java
// AbstractStringBuilder.shift

private void shift(int offset, int n) {
    System.arraycopy(value, offset << coder,
                        value, (offset + n) << coder, (count - offset) << coder);
}
```

`System.arraycopy` copies a source array at a position, to a destination array from a position.

## How does StringBuilder reverse a string?

So how does the `StringBuilder` class reverse a string?

```java
public AbstractStringBuilder reverse() {
    byte[] val = this.value;
    int count = this.count;
    int n = count - 1;
    if (isLatin1()) {
        for (int j = (n-1) >> 1; j >= 0; j--) {
            int k = n - j;
            byte cj = val[j];
            val[j] = val[k];
            val[k] = cj;
        }
    } else {
        StringUTF16.reverse(val, count);
    }
    return this;
}
```

In this article I am only looking at the `isLatin1()` branch of the conditional. This is where most english words would fall into as each character is 1 byte long. However, I would imagine _in the real world_ the condition would likely fall into the `else` block. 
I really like this code - particularly the `for` loop. But why?

```java
for (int j = (n-1) >> 1; j >= 0; j--) {
    int k = n - j;
    byte cj = val[j];
    val[j] = val[k];
    val[k] = cj;
}
```

More often than not, programmers will iterate over the full length of the string to reverse it - including my solution, but that is not what is happening here with this bitwise operation.

```java
int j = (n-1) >> 1
// if n = 10, then j = 5
// if n = 25, then j = 12
// if n = 99, then j = 49
```

This bit shifting finds the middle byte `j`. If the string letter count is even, `k` will be the same as `j`, otherwise it will be the byte to the right position. A temporary variable `cj` is used to stage the value for the swap.

```java
byte cj = val[j];
val[j] = val[k];
val[k] = cj;
```

So if a string 10 characters long, there are 5 iterations through the loop. If it is 11, there will stil be 5 iterations. If the string is 12 characters long it will be 6 iterations... and so on.

This is tight code in terms of performance and speed. It doesn't do any unnecessary allocation, doesn't loop over the entire string, and does a bitwise right shift instead of a divide by two. However, the bitwise shift doesn't necessarily mean increased perf on some CPUs, but for a Java standard library function, this is a good approach where the runtime isn't clear and aligns with the _write once, run anywhere_ mantra. I suspect that is why I see this approach sometimes in library code, but almost never in application code.

I'm not sure this solution would fly in an interview however. A lot of information systems developers might not know a lot about bit shifting, and some may prefer functional approaches such as using only immutable data structures. It would probably be best to ask the interviewer for clarification on the trade-offs acceptable and gauge a sense of their biases and preferences. Are you writing something for your fellow programmers, or are you trying to create the fastest `reverse` function in town? These yield different solutions.

## What on earth did I just read?

On looking back, although my solution has an non-obvious buffer shift, I still think it is a clean and tight way of solving the problem in Kotlin _or_ Java, partically when being timed and having a stranger staring over your shoulder scrutinizing your code. It is immediately clear to a reader what is happening, and takes into consideration what is happening inside the `StringBuilder` class. It is thread-safe in the sense that a new StringBuilder object is allocated per invocation of the method and doesn't hoist any state to the class level - although StringBuilder itself is not thread-safe, see `StringBuffer` instead.

This was just one mans retrospective of a fun interview question. Maybe it's software engineering rigour, but it might also be bikeshedding. I will let you decide. 


Favourite Footguns Pt 3: No resets in Object Pools


On the JVM, short-lived objects are inexpensive to allocate and collect. Garbage Collection (GC) is one of those topics engineers celebrate for convenience and safety, which in many cases increases development speed. There are, however, classes of memory-conscious programming where high allocation and GC pauses are undesirable - think Low-Latency Trading and computations on IoT sensor data. Although not a silver bullet solution, Object Pools, a type of creational design pattern, can provide constant time access to objects, increase the overall throughput of the application, and reduce allocation footprint leading to a very different heap profile. They are useful when instantiation is costly - examples include XML Parsers, Video Game Objects, and Database Connections.

In short, Object Pools are a collection of mutable objects that can be dequeued and enqueued rather than creating a new object and discarding it for garbage collection. However, it is easy to footgun the design of the Object Pool by not resetting the resident objects.

### An Example

For this example, I am using **Java 17**, with **Lombok 18** for syntactical convenience. As this post is just to demonstrate how you can shoot yourself in the foot rather than build out something production-grade, we will only be pooling a very simple Object - a Person with two properties (name + age) and a public method to display these properties nicely.

```java
package com.pjhampton.objpool;

import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;

@Getter
@Setter
@NoArgsConstructor
final public class Person {

    // Contrived and simple object to pool
    
    private String name;
    private Integer age;

    public void sayHello() {
        System.out.printf("Hi, I'm %s, and I'm %d!%n%n", name, age);
    }
}
```

Below I create a no-thrills object pool that accepts a generic type. In this blog post, it will just be kept to a Person object though. An ```ArrayDeque<T>``` is used as the internal collection with a create function to build the Pool. The reason why I call this a no-thrills pool is that more sophisticated implementations can resize based on supply and demand parameters (hence the use of `ArrayDeque<T>` here) and do resetting (which we will get to). On a side note, this object pool has First In First Out (FIFO) semantics, but you could easily make it LIFO, or use a combination of the two (for whatever reason).

```java
package com.pjhampton.objpool;

import lombok.NoArgsConstructor;

import java.util.ArrayDeque;
import java.util.List;

@NoArgsConstructor
final public class Pool<T> {

    private ArrayDeque<T> pool;

    public void create(List<T> objects) {
        pool = new ArrayDeque<>(objects);
    }

    public T fetchOne() {
        return pool.pollFirst();
    }

    public void putOne(T object) {
        pool.addLast(object);
    }
}
```

In the main class, I create a 4-person object pool and run a 100 iteration for loop. This loop sets up the objects properties and uses the sayHello() function to print it's internal state in a friendly way to STDOUT. 

```java
package com.pjhampton.objpool;

import java.util.ArrayList;
import java.util.List;

public class Main {

    final private static Pool<Person> objectPool = new Pool<>();

    private static void initializeObjectPool() {
        List<Person> personList = new ArrayList<>();

        // Add 5 objects to the pool
        for (int i = 0 ; i < 5 ; i++) {
            var person = new Person();
            personList.add(person);
        }

        objectPool.create(personList);
    }

    public static void main(String[] args) {
        initializeObjectPool();

        for (int i = 0 ; i < 100 ; i++) {
            var person = objectPool.fetchOne();

            // Set up Person
            person.setName(String.format("Person %d", i));
            person.setAge(i + 10);

            // Print hash code + person message
            System.out.println(person.hashCode());
            person.sayHello();

            objectPool.putOne(person);
        }
    }
}
```

The printed output looks like so:

```txt
1442407170
Hi, I'm Person 0, and I'm 10!

1028566121
Hi, I'm Person 1, and I'm 11!

1118140819
Hi, I'm Person 2, and I'm 12!

1975012498
Hi, I'm Person 3, and I'm 13!

1808253012
Hi, I'm Person 4, and I'm 14!

1442407170 <-- First object reused again - great success 💅
Hi, I'm Person 5, and I'm 15!

1028566121
Hi, I'm Person 6, and I'm 16!

...
```

### The Footgun

In this example, things can't really go wrong. After all - we are in control of what the Person properties are. We set them to be `name = Person $i`, and `age = $i + 10`. But if the data was coming in from an external source that the system doesn't control (which is usually the case in any useful system), we could run into a data issue very quickly. For example, let's assume that every other Person does not have an age. This programs output would start to look like so:

```java
public static void main(String[] args) {
    initializeObjectPool();

    for (int i = 0 ; i < 100 ; i++) {
        var person = objectPool.fetchOne();

        // Set up Person
        person.setName(String.format("Person %d", i));

        // implement bug to skip setting a persons age
        // pretend it is missing on upstream source 😉
        if (i % 2 == 0)
            person.setAge(i + 10);

        // Print hash code + person message
        System.out.println(person.hashCode());
        person.sayHello();

        objectPool.putOne(person);
    }
}
```

There is now an obvious data issue - old data on the objects is being recycled when the object is being taken out of the pool again.

```txt
1442407170
Hi, I'm Person 0, and I'm 10!

1028566121
Hi, I'm Person 1, and I'm null!

1118140819
Hi, I'm Person 2, and I'm 12!

1975012498
Hi, I'm Person 3, and I'm null!

1808253012
Hi, I'm Person 4, and I'm 14!

1442407170 <-- oh boi. We are using the age from Person 1 😅
Hi, I'm Person 5, and I'm 10!

1028566121
Hi, I'm Person 6, and I'm 16!

```

### Solution: In-band or Out-of-Band Resets

The solution is to implement in-band or out-of-band object resets. I can't think of a time when you wouldn't reset, but regretfully I have come across an implementation that didn't and has insprired me to write this post. It's not that straightforward though as resetting could increase latency indirectly - so maybe partial resets are ideal for some pooling strategies.


We can do this reset or zeroing out when we take it from the pool or when we add it back in. 

```java
public interface PoolObject {

    void reset();
}
```

```java
final public class Person implements PoolObject {

    ...
    public void reset() {
        name = null;
        age = null;
    }
}
```

```java
final public class Pool<T extends PoolObject> {

    ...

    // ✅ In Band reset
    public T fetchOne() {
        var obj = pool.pollFirst();
        obj.reset(); // could blow if pool empty
        return obj;
    }

    // ✅ Out of Band reset
    public void putOne(T object) {
        object.reset();
        pool.addLast(object);
    }
}
```

Obviously, you wouldn't do both - but either one should suffice in a lot of use cases. I would be interested in profiling to see if there is much of a difference. I guess that will be for another day.


Grokking Sed


SED(1) - a handy *nix tool for stream editing which enables users to manipulate text within files. Many developers will have come across an example like this before, as seen on the sed man page:

```bash
sed -i '' -e 's/foo/bar/g' test.txt
```

This above line of code **replaces all occurances of 'foo' with 'bar' in a file without create a backup** of the original file. That is the extent most developers will ever use sed. I usually come across it as an agressive way of substituting values in a config file or script of some sort. sed is a little more interesting that just that though. This post looks at some a few additional ways to use sed for fun and profit*.

## Backing up

In the past, I have found when I use sed with regular expressions the outcome can sometimes be perplexing. Debugging regular expressions can be a tedious process that takes patience and skill.

```bash
sed -i.bak 's/foo/bar/g' test.txt
```

The above command will modify the file in place and make a copy of the file in place named **test.txt.bak**. 

## Addressing

Addressing is what makes sed so powerful. If you are just looking to execute your replacement on a specific line you can prefix with the line number.

```bash
sed -i '/l244/foo/bar/' file.txt
```

The above command will only do a subtitution on line 244 of the file.

```bash
sed -i '/hello/s/foo/bar' file.txt
```

You can also just replace text based on lines containing the word 'hello' and inverse this with the bang operator, where you replace foo with 'bar' on lines that **do not** contain the word 'hello'

```bash
sed -i '' -e '/hello/!s/foo/bar/' file.txt
```

## Scripts

Sed commands can grow out of control pretty fast. You can find yourself in a situation where you are pipeing sed commands into sed commands like the below when executing multiple substitutions:

```bash
sed -i 's/foo/bar/g' /etc/service/my.conf | sed -i 's/bar/baz/g'
```

A cleaner way to write the above would be chaining substitution arguments with the **-e** flag

```bash
sed -e 's/foo/bar/g' -e 's/bar/baz/g' -i.bak /etc/service/my.conf
```

However, if you have many of these commands (my personal rule of thumb is ~4) is to use a dedicated script. These commands can be grouped together and run as a file - example_script.sed

```perl
s/foo/bar/g
s/bar/baz/g
```

and run together with the **-f** flag:

```bash
sed -f example_script.sed /etc/service/my.conf
```

## Congratulations! 🦄

You just grokked sed!

\*No profit promised.


Favourite Footguns Pt 2: Leaking secrets


A practice that gets a lot of attention these days is mishandling secrets and other sensitive material. When I moved from the finance industry into security I started thinking about better ways to write _secure code_. I have come to learn that secret management is a tricky business to get correct 100% of the time. Because of this, I am now operating under the practice that secrets will leak, and it really sucks when it happens. The art here is motivating people enough to fix the leak, promote a culture of secure coding and leveraging state-of-the-art infrastructure to _manage_ (rather than just _store_) sensitive material. Accompanying code in this post is written in Scala (Dotty) 3.1.0 (the latest release at time of writing).

### Thought 1: Secret as a Type

Assuming you have access to a strongly typed programming language like Scala, you can wrap a String (or really any type) as a class to wrap the value and mask it if a caller tries to send it to a log or stream. 

```scala
class Secret(private val secret: String) {
  def get = secret
  override def toString = "********"
}

object SecretsExample extends App {
  val secret = Secret("this is so secret 🤐")
  println(s"I have a secret: $secret")
}
```

### Thought 2: Read-Once Objects

I like the idea of only allowing a value to be read once from an object, and subsequent readers get informed of a message explaining that it has already been read. A good way of doing this is returning a Tuple/Pair like structure with the secret in the left element, and a message in the right element to communicate why the secret is null. This is a good strategy for combating unintended use by programmers.

```scala
import java.util.concurrent.atomic.AtomicReference

class ReadOnceValue(value: AtomicReference[String]) {
  
  def get: (String, String) = {
    val secret = value.getAndSet(null)
  
    if (secret == null) {
      return (null, "Read-once value has already been returned")
    } else {
      return (secret, null)
    }
  }
}

object Main extends App {
  val sensitive = ReadOnceValue(AtomicReference("p4ssword"))
  println(sensitive.get) // ==> (p4ssword,null)
  println(sensitive.get) // ==> (null,Read-once value has already been returned)
  println(sensitive.get) // ==> (null,Read-once value has already been returned)
}
```

### Thought 3: Failing on unintended use

Doubling down on the previous thought, another strategy I think that is quite effective is failing on unintended use by another part of the application. This is easy to do by extending the java.io.Externalizable interface and throw an exception on unintended serialization. 

```scala
import java.io.{Externalizable, FileInputStream, FileOutputStream, ObjectInput, ObjectOutput, ObjectOutputStream}
import java.util.concurrent.atomic.AtomicReference

class ReadOnceValue(value: AtomicReference[String]) extends Externalizable {
  
  def get: (String, String) = {
    val secret = value.getAndSet(null)
  
    if (secret == null) {
      return (null, "Read-once value has already been returned")
    } else {
      return (secret, null)
    }
  }
  
  override def readExternal(in: ObjectInput): Unit = fail()

  override def writeExternal(out: ObjectOutput): Unit = fail()

  def fail(): Unit =
    throw UnsupportedOperationException("Not permitted on sensitive value")
}

object Main extends App {
  val sensitive = ReadOnceValue(AtomicReference("p4ssword"))
  
  val outStream = ObjectOutputStream(FileOutputStream("/tmp/tmpfile.txt"))
  outStream.writeObject(sensitive) // ==> 💥 Throws UnsupportedOperationException
  outStream.close()
}
```

### Thought 4: Handle Errors / Exceptions carefully

When I was younger and first using the internet I used to get a lot of 'The Webmaster has been notified' messages when a website fell over for some sort of reason. Then I started seeing a lot of blank pages with status codes and exception messages that I didn't understand as teenager. The internet seems to have grown up, which is great, but some applications continue to serve users with exception messages which can reveal system information to a bad actor. It can communicate the internal structure of an application such as the language, libraries, class heirarchy and can arm a malacious actor with enough information to exploit the application and access sensitive resources such as databases, LDAP servers, sensitive network information. 

Exceptions can be great for developers (personal opinion), but they are not a good user experience. Chances are a user won't know what to do with the information, kind of like me when I was younger. From experience, I think traping exceptions to avoid leaking sensitive data by proxy is a good idea. 

```scala
def iCanFail(name: String): String = {
  if (name != "pete") {
    throw Exception("You are not pete")
  }
  
  return s"Hi, ${name}"
}

object ExMain extends App {
  
  try {
    println(iCanFail("sarah"))
  } catch {
    case e => {
      // Log to inform dev
      logger.error("Failed to handle input", e)
      // Provide a better user experience with explanation
      println("Sorry, our service can't handle your request right now. We're on it!")
    } 
  }
}
```

### Thought 5: Obfuscation by Symmetric Encryption

Encryption can be thought of as a 2 way pure function that can mask data. It can be accomplished using a shared key (Symmetric) or Public/Private key pair (Asymmetric). The JCA allows programmers to use cryptographic functionality such as Secure Random, Hashing Algos, Sym/Asym Enryption and more. The implementation of the JRC is left up to the JRE vendors to create their own implementation and multiple algorithm implementations may exist on the classpath at one time. I don't particularly like this method as the secret key tends to get shared around multiple programmers and then there is a trust factor and human accidental leak risk.

```scala
import java.security.SecureRandom
import javax.crypto.spec.{IvParameterSpec, SecretKeySpec}
import javax.crypto.{Cipher, KeyGenerator, SecretKey}
import org.bouncycastle.util.encoders.Hex

class SymEncrypt {

  private val aes = "AES"
  private val cipherAlgo = "AES/CBC/PKCS5Padding"
  
  def createAESKey(): SecretKey = {
    val secureRandom = SecureRandom()
    val keyGenerator = KeyGenerator.getInstance(aes)
    keyGenerator.init(256, secureRandom)
    return keyGenerator.generateKey()
  }
  
  // I have seen developers hardcode the initialization vector in code so they don't need
  // to store as a separate secret. 
  def createInitializationVector(): Array[Byte] = {
    val secureRandom = SecureRandom()
    val initializationVector = new Array[Byte](16)
    secureRandom.nextBytes(initializationVector)
    return initializationVector
  }
  
  def aesEcrypt(plainText: String, secretKey: SecretKey, initializationVector: Array[Byte]): Array[Byte] = {
    val cipher = Cipher.getInstance(cipherAlgo) 
    val ivParameterSpec = IvParameterSpec(initializationVector)
    cipher.init(Cipher.ENCRYPT_MODE, secretKey, ivParameterSpec)
    return cipher.doFinal(plainText.getBytes())
  }
  
  def aesDecrypt(cipherText: Array[Byte], secretKey: SecretKey, initializationVector: Array[Byte]): String = {
    val cipher = Cipher.getInstance(cipherAlgo)
    val ivParameterSpec = IvParameterSpec(initializationVector)
    cipher.init(Cipher.DECRYPT_MODE, secretKey, ivParameterSpec)
    val decryptedText = cipher.doFinal(cipherText)
    return String(decryptedText)
  }
}

// Bouncy Castle on classpath
// https://mvnrepository.com/artifact/org.bouncycastle/bcprov-jdk15on
// libraryDependencies += "org.bouncycastle" % "bcprov-jdk15on" % "1.70"
object Main extends App {
  
  // Generate AES (Symmetric) encryption key
  val enc = SymEncrypt()
  val key = enc.createAESKey()
  println(Hex.toHexString(key.getEncoded()))

  val initVector = enc.createInitializationVector()
  val sensitiveText = "End programming language culture wars!"
  
  // Symmetrically enrypt some plain text
  val encryptedText = enc.aesEcrypt(sensitiveText, key, initVector)
  println(Hex.toHexString(encryptedText))
  
  // Symmetrically decrypt some plain text
  val decryptedText = enc.aesDecrypt(encryptedText, key, initVector)
  println(decryptedText)
}
```

### Thought 5: Obfuscation by Asymmetric Encryption

A more secure way of encrypting data is with asymmetric encryption. Instead of a shared key, data is encrypted with a private key (not shareable) and decrypted with a public key (sharable with sources you trust). Keys can be managed securely by hardware or services such as AWS KMS.

```scala
import java.security.{KeyPair, KeyPairGenerator, PrivateKey, PublicKey, SecureRandom}

import javax.crypto.Cipher
import org.bouncycastle.util.encoders.Hex

class AsymEncrypt {

  private val rsa = "RSA"
  
  def createRSAKeyPair(): KeyPair = {
    val secureRandom = SecureRandom()
    val keyPairGenerator = KeyPairGenerator.getInstance(rsa)
    keyPairGenerator.initialize(4096, secureRandom)
    return keyPairGenerator.generateKeyPair()
  }
  
  // Encrypt with the private key
  def rsaEncrypt(plainText: String, privateKey: PrivateKey): Array[Byte] = {
    val cipher = Cipher.getInstance(rsa)
    cipher.init(Cipher.ENCRYPT_MODE, privateKey)
    return cipher.doFinal(plainText.getBytes())
  }
  
  // Decrypt with the public key
  def rsaDecrypt(cipherText: Array[Byte], publicKey: PublicKey): String = {
    val cipher = Cipher.getInstance(rsa)
    cipher.init(Cipher.DECRYPT_MODE, publicKey)
    val result = cipher.doFinal(cipherText)
    return String(result)
  }
}

// Bouncy Castle on classpath
// https://mvnrepository.com/artifact/org.bouncycastle/bcprov-jdk15on
// libraryDependencies += "org.bouncycastle" % "bcprov-jdk15on" % "1.70"
object Main extends App {

  // Generate RSA (Asymmetric) encryption key pair
  val enc = AsymEncrypt()
  val keyPair = enc.createRSAKeyPair()
  println(s"Private Key ${Hex.toHexString(keyPair.getPrivate().getEncoded)}")
  println(s"Public Key  ${Hex.toHexString(keyPair.getPublic().getEncoded)}")

  val sensitiveText = "All day and night I drink coffee"
  println(s"Original Text: ${sensitiveText}")
  
  // Asymmetrically enrypt some plain text
  val encryptedText = enc.rsaEncrypt(sensitiveText, keyPair.getPrivate())
  println(s"Encrypted Text: ${Hex.toHexString(encryptedText)}")

  // Asymmetrically decrypt some plain text
  val decryptedText = enc.rsaDecrypt(encryptedText, keyPair.getPublic())
  println(s"Decrypted Text: ${decryptedText}")
}
```

### Thought 6: Mishandling heap dumps

An interesting thought I had while concluding this post is what is secrets could be leaked via a heap dump as most of the secrets will be held in memory - very likely on the heap. That would be truely the worst way to get pwned. Having a management and deletion culture of heap dumps, and a secure way or getting and storing them would be ideal.

### Parting thoughts

There is no _one way_ to manage sensitive material, but there are a few steps that can be taken to mitigate the potential leaks and attack avenues. I think it is important to be cognizant of best practices, cultivate a culture of care and pave the path for other developers not to leak secrets indirectly. 


Elasticsearch with Kotlin

Rust borrow checker notes


In C++, a programmer must explicitly allocate and free memory. Around C++11/C++14 the concept of smart pointers were enhanced with `std::shared_ptr` and `std::unique_ptr` to allocate in the _Resource acquisition is initialization_ paradigm (RAII). This is a *runtime* construct that allows the deleter to be called once the the last pointer of that object goes out of scope - they perform this memory management by automatically tracking references. This is helpful for dealing with dangling pointers and other memory leaks caused by not freeing up heap. 

```cpp
#include <iostream>
#include <memory>
#include <string>

class Person {

public:
    Person() : name("Nobody") {}
    Person(const std::string& seedName) : name(seedName) {}

    void setName(const std::string& newName) {
        name = newName;
    }

    void greet() {
        std::cout << "Hello " << name << std::endl;
    }

private:
    std::string name;
};
```


```cpp
int main() {

    std::unique_ptr<Person> person = std::make_unique<Person>();
    person->setName("Emily");
    person->greet();

    // deleter called

    return 0;
}
```

When I first heard about Rust and the Borrow Checker I thought it sounded very similar to the above - C++ smart pointers. Conceptually, they seem to serve similar purposes but take memory safety to another level with static typing and compile-time checks. 

## Rust

The [Rust programming langauge](https://www.rust-lang.org/) does things a little differently and manages memory through concept of _ownership_ and _borrowing_. Even though I have experience with smart pointers and move semantics in C++, I still found the concepts of the Rust Borrow checker to be a little tricky to grok. In summary:

- **Ownership**: Each value in Rust has a single "owner", ensuring safe memory management.
- **Borrowing**: Values can be borrowed either mutably or immutably.
- **Exclusive Mutability**: You can have either one mutable reference or multiple immutable references to a value, but not both. This helps with thread-safety.
- **Scope-based Lifetimes**: References to values cannot outlive the values they point to.
- **Compile-time**: The borrow checker operates at compile time, ensuring no runtime overhead while maintaining safety.

### Example 1

In this first example I create a mutable string `s` and create 2 immutable borrows by referencing the string. This is legal as it can't be changed. However, if I were to borrow it as a mutable reference, the borrow checker would fail. I can create a mutable reference to the variable after immutable references are no longer in scope. 

```rust
fn main() {
    let mut s = String::from("Emily");

    let r1 = &s; // Immutable borrow.
    let r2 = &s; // Immutable borrow.

    // let r3 = &mut s; // Error! Can't borrow `s` as mutable because it's already borrowed as immutable.

    println!("{} / {}", r1, r2);

    // r1 and r2 are no longer used after this point.

    let r3 = &mut s; // Mutable borrow is allowed now.
    println!("{}", r3);
}
```

To carry on from this example you can have one mutable reference to a variable, and once that goes out of scope it is possible to create another mutable reference.

```rust
fn main() {
    let mut s = String::from("Emily");

    let r1 = &mut s;  // First mutable borrow starts here.
    // let r2 = &mut s;  // Error! Can't have two mutable borrows at once.

    println!("{}", r1);

    let r2 = &mut s;  // This is allowed as the first mutable borrow ended.
    println!("{}", r2);
}
```

### Example 2

In this next example, the borrow checker prevents me from pushing an integer to a vector once an immutable reference to `nums` is created. This prevents a separate part of the program from mutating the structure from another part of the program, potentially facing awkward race conditions or data corruption. One the reference goes out of scope the caller can mutate the vector.

```rust
fn main() {
    let mut nums = vec![1, 2, 3, 4, 5];

    let first = &nums[0];  // Immutable borrow starts.

    // nums.push(6); // Error! Can't mutate `nums` while it's borrowed.

    println!("First number: {}", first);

    nums.push(6);  // It's okay now to mutate `nums` as the borrow has ended.
}
```

### Example 3

In this example the borrow checker prohibits iterating over an array and mutating it at the same time. I can't think of a time when someone would do this - as it seems pretty dumb. But it's nice to know this can be caught at compile time and probit the developer from doing this in the first place.

```rust
#[allow(unused_mut)]

fn main() {
    let mut vector = vec![1, 2, 3, 4, 5, 6];

    for n in &vector {
        // Error! Can't mutate `vector` while it's being iterated over immutably.
        // vector.push(7);
        println!("{}", n);
    }
}
```

### Example 4

This is a slightly more contrived example, but I believe it could happen if you were writing Rust with an imperitive mindset. A `struct Person` is created with a `get_name` function that returns the name property. However, within a local scope, the `name` property reference cannot outlive the `Person struct` instance, and therefore hoisting the value to a higher scope wouldn't happen.

```rust
struct Person {
    name: String,
}

fn get_name<'a>(person: &'a Person) -> &'a str {
    &person.name
}

fn main() {
    let name;
    {
        let person = Person { name: String::from("Emily") };
        name = get_name(&person);
        println!("{}", name)

        // Error! `name` outlives `person`. 
        // Thus, the reference held by `name` would be dangling.
    }
    
    // println!("{}", name); // would never happen
}
```

## Final thoughts

This surface look at the Rust borrow checker is pretty interesting - I particullary like the compiler output when there are referencing issues. It's clear that simple, but common manual memory-management issues in C/C++ are addressed by the borrow checker such as null pointer deferencing, data races and dangling pointers. The overall syntax and package management of the language is also very refreshing. I get the hype.


IPC - Sharing memory between JVMs


Oftentimes, software need to talk to other software - this is where IPC (Inter-Process Communication) comes in if they are located on the same physical resource. They can take advantage of numerous messaging fabic such as message queues, sockets, HTTP, UDP, and countless others - many I have never heard of! Utilising shared memory is one of the fastest ways of of passing data between processes (Between JVMs or others) on the same machine because it's stored virtually in memory. It is not limited to the JVM heap space - it's limited by disk space and can run orders of magnatiude faster than traditional messaging passing frameworks. In this example I present a **1 producer / 1 consumer** setup where data is written to a memory-mapped file, and read and printed. This is the simplist example I could come up with.

## Consumer

Let's write the consumer as that is what we will run first. First, I constrain the message buffer to 1024 Bytes and then open a `RandomAccessFile` called `smem.dat` (shared memory dot data). The `RandomAccessFile` acts as a large array of bytes stored in the file system. `RandomAccessFile.getChannel()` returns a reading/writing channel to that file. 

```java
channel.map(FileChannel.MapMode.READ_WRITE, 0, MESSAGE_SIZE);
``` 

The above snippet maps a region of file directly into memory - it allows us to work with the file as if it were a large array of bytes in memory. This is powerful as it allows us to read and write to the file directly, rather than traditional slow disk operations. The actual mechanics of how the data is transferred between the program and the file are handled by the operating system and although is incredibly efficient, can be subject to things like virtual memory management and paging.

In the while loop that polls for new messages, there is a syncronization mechanism in place `buffer.get(MESSAGE_SIZE - 1) == (byte) 1` where the producer will signal to the consumer that a new message is available by setting the last byte to `1`. The message is then read out of the buffer based on the length of the message and decoded as a string for printing. At the end, the consumer will set `buffer.put(MESSAGE_SIZE - 1, (byte) 0);` to signal to the producer that the massage has been read and is ready for the next message.

```java
import java.io.RandomAccessFile;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;

public class Consumer {

    private static final int MESSAGE_SIZE = 1024;

    public static void main(String[] args) throws Exception {
        try (RandomAccessFile file = new RandomAccessFile("smem.dat", "rw");
             FileChannel channel = file.getChannel()) {

            MappedByteBuffer buffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, MESSAGE_SIZE);

            while (true) {
                if (buffer.get(MESSAGE_SIZE - 1) == (byte) 1) {
                    int length = buffer.getInt(0);
                    byte[] bytes = new byte[length];
                    buffer.position(4);
                    buffer.get(bytes);
                    String message = new String(bytes);

                    System.out.println("Consumed: " + message);

                    buffer.put(MESSAGE_SIZE - 1, (byte) 0);
                }
            }
        }
    }
}
```

## Producer

The producer in this example is not too different from the consumer. It create a channel to the `RandomAccessFile` called smem.dat and maps it to memory. However, the producer is stateful in the sense it holds an integer value `0` and increments it for every message it recieves. `buffer.clear()` and `buffer.putInt(0, bytes.length);` resets the buffers position handle to the start and `buffer.putInt(0, bytes.length);` is used to write the length of the message at the beginning of the buffer (position 0). This is important because the consumer will need to know how long the message is when it reads it. `buffer.position(4)` moves the buffer position to 4 to write the message - this is because an `int` in Java is 4 Bytes (again, it's the length of the message).

Just like the consumer `buffer.put(MESSAGE_SIZE - 1, (byte) 1);` sets the ready flag to signal to a consumer that a new message is available. `buffer.force();` is finally called to ensure the changes are written to the memory-mapped file as changes are not garunteed to be written until the buffer is garbage collected - which I was surprised to learn, but it's true!

```java
import java.io.RandomAccessFile;
import java.nio.ByteBuffer;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;

public class Producer {

    private static final int MESSAGE_SIZE = 1024;

    public static void main(String[] args) throws Exception {
        try (RandomAccessFile file = new RandomAccessFile("smem.dat", "rw");
             FileChannel channel = file.getChannel()) {

            MappedByteBuffer buffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, MESSAGE_SIZE);
            int i = 0;

            while (true) {
                String message = "Msg-" + i++;
                byte[] bytes = message.getBytes();

                buffer.clear();
                buffer.putInt(0, bytes.length);
                buffer.position(4); 
                buffer.put(bytes);
                buffer.put(MESSAGE_SIZE - 1, (byte) 1);
                buffer.force();

                System.out.println("Produced: " + message);
            }
        }
    }
}
```

For demonstration purposes, I recorded a tmux session running one producer, and a consumers - the same consumer is split across two windows. It was run on a MacBook Air M1 2020 with 8GB memory running Ventura 13.0.1. 

![ipctmux](https://github.com/pjhampton/pjhampton/assets/8960296/8811ce5a-7a71-405a-8817-05f55a7b4001)


Favourite Footguns Pt 4: Not flipping a ByteBuffer


`java.nio.ByteBuffer` is one of my favourite classes to work with on the JVM. You can use it for manipulating data in memory, such as byte-level data structures, reading from and writing to sockets, and parsing data received from a network. It can also be created with different types of backing arrays and can also be used with memory-mapped files. It is also nice because APIs are provided to allocate buffer both on and off the JVM heap. However, weird things can happen if you don't flip the buffer. Here is an example:

```java
public class BBNoFlip {

    public static void main(String[] args) {

        // Encode a message into a ByteBuffer
        String message = "Long live Duke!";
        ByteBuffer buffer = ByteBuffer.allocate(1024);
        buffer.put(message.getBytes(StandardCharsets.UTF_8));

        // Decode the message from the ByteBuffer
        byte[] messageBytes = new byte[buffer.remaining()];
        buffer.get(messageBytes);
        String decodedMessage = new String(messageBytes, StandardCharsets.UTF_8);

        System.out.println("Original: " + message);
        System.out.println("Decoded: " + decodedMessage);
    }
}
```

The output is as follows (_decoded_ truncated for this post):

```bash
Original: Long live Duke!
Decoded:                                   
```

The String `Long live Duke!` is 15 bytes, and we allocated 1024 bytes of Buffer space onto the JVM via `ByteBuffer.allocate`. When we read the data back out of the buffer into a new byte array, we get 1019 bytes of NULL or garbage (whatever happens to be in the buffer) and our encoded data. This could be problematic if you are reading data off a network connection and writing it to a database of somesort.

The remedy is `ByteBuffer.flip`. When we call `ByteBuffer.flip` we set the limit to the current position and reset the position back to 0. 

```java
public class BBFlip {

    public static void main(String[] args) {

        // Encode a message into a ByteBuffer
        String message = "Long live Duke!";
        ByteBuffer buffer = ByteBuffer.allocateDirect(1024);
        buffer.put(message.getBytes(StandardCharsets.UTF_8));

        // flip the buffer (obviously)
        buffer.flip();

        // Decode the message from the ByteBuffer
        byte[] messageBytes = new byte[buffer.remaining()];
        buffer.get(messageBytes);
        String decodedMessage = new String(messageBytes, StandardCharsets.UTF_8);

        System.out.println("Original: " + message);
        System.out.println("Decoded: " + decodedMessage);
    }
}
```

Success 🙌 The data was written into the buffer and decoded with much success, and feet in tact. 


```bash
Original: Long live Duke!
Decoded: Long live Duke!
```


ArgoCD Image Updater - GCR Connectivity


ArgoCD is a fantastic piece of software in my opinion. It's a brilliant tool for syncing Kustomize and/or Helm projects, or simple file manifest directories with a Kubernetes cluster. In a recent proof-of-concept project of moving our teams Kafka processor workloads onto Googles Kubernetes Engine (GKE) we made the decision early on to auto-deploy the Kafka Processors once they make it through a series of CI checks - we have a toil and throughput problem so this may seem extreme to some, but we know we can dial it back if required. 

An annoying caveat is ArgoCD doesn't support container image updating natively ([yet](https://github.com/argoproj-labs/argocd-image-updater/issues/491)). At the time of writing this post, my argo / k8s setup configuration looked like so:

```yaml
$ argocd version

argocd: v2.5.7+e0ee345.dirty
  BuildDate: 2023-01-18T04:25:01Z
  GitCommit: e0ee3458d0921ad636c5977d96873d18590ecf1a
  GitTreeState: dirty
  GoVersion: go1.19.5
  Compiler: gc
  Platform: darwin/arm64
argocd-server: v2.6.6+6d4de2e
  BuildDate: 2023-03-16T22:25:45Z
  GitCommit: 6d4de2ec5d49fa2c6823f2b7d101607a839be3fa
  GitTreeState: clean
  GoVersion: go1.18.10
  Compiler: gc
  Platform: linux/amd64
  Kustomize Version: v4.5.7 2022-08-02T16:35:54Z
  Helm Version: v3.10.3+g835b733
  Kubectl Version: v0.24.2
  Jsonnet Version: v0.19.1
```

Until `argocd-image-updater` is merged into ArgoCD ([See this proposal](https://github.com/argoproj/argo-cd/pull/10447#issuecomment-1333839258)) you will need to set it up as documented, however - I ran into a perms issue when connecting it to Google Cloud Registry (GCR). After installing and getting everything set up, we were running into this specific problem 

```python
DEBU[0000] Creating in-cluster Kubernetes client        
INFO[0000] retrieving information about image            image_alias= image_digest= image_name=gcr.io/testtest/k8s-test image_tag=latest registry_url=gcr.io
WARN[0000] Target platform is 'darwin/arm64', but that's not a supported container platform. Forgot --platforms? 
DEBU[0000] setting rate limit to 20 requests per second  prefix=gcr.io registry="https://gcr.io"
DEBU[0000] Inferred registry from prefix gcr.io to use API https://gcr.io 
INFO[0000] Fetching available tags and metadata from registry  application=test image_alias= image_digest= image_name=gcr.io/testtest/k8s-test image_tag=latest registry_url=gcr.io
FATA[0001] could not get tags: denied: Failed to read tags for host 'gcr.io', repository '/v2/testtest/k8s-test/tags/list'  application=test image_alias= image_digest= image_name=gcr.io/testtest/k8s-test image_tag=latest registry_url=gcr.io
```

## How I worked around this limitation

First, create a service account in Google Cloud with the following permissions:

```yaml 
- Storage Admin
- Container Registry Service Agent
```

Then add a key and download in JSON format. It should look something like so:

```json
{
  "type": "service_account",
  "project_id": "testtest",
  "private_key_id": "1111111111111111111111111111111111111111",
  "private_key": "-----BEGIN PRIVATE KEY-----\n-----END PRIVATE KEY-----\n",
  "client_email": "your-service-account@testtest.iam.gserviceaccount.com",
  "client_id": "1111111111111111111",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/your-service-account%40testtest.iam.gserviceaccount.com"
}
```

You are then clear to create a secret in your Kubernetes namespace. It should be in this format: `<username>:<password>` where `<username>=_json_key` and the password is the minified jsonpayload. No surrounding quotes required. 

```yaml
apiVersion: v1
kind: Secret
metadata:
  namespace: argocd
  name: image-updater-gcr
type: Opaque
stringData:
  secret: _json_key:{"type":"service_account","project_id":"testtest", ...}
```

Once the secret is applied with:

```sh
$ kubectl apply -f secret.yml -n argocd
```

You can update the ConfigMap for `argocd`

```sh
$ kubectl -n argocd edit cm argocd-image-updater-config
```

With the following configuration

```yaml
apiVersion: v1
kind: ConfigMap
data:
  log.level: debug
  registries.conf: |
    registries:
    - name: Google Container Registry
      prefix: gcr.io
      api_url: https://gcr.io
      default: true
      credentials: secret:argocd/image-updater-gcr#secret
```

To set up your strategy you can add the [following annotations](https://argocd-image-updater.readthedocs.io/en/stable/basics/update-strategies/#strategy-digest) to your Argo application. Below is an example of the daemon process checking the sha digest of the `latest` tagged image. 

```yml
@argocd-image-updater.argoproj.io/image-list: myimg=some/image:latest
@argocd-image-updater.argoproj.io/myimg.update-strategy: digest
```

Hopefully this post saves you a couple of hours, whoever you are.


Doing more with less time


A company I used to work for introduced a git-spying tool to report on the quantity and quality of the code a programmer produced - I'm not sure how it would evaluate quality as quality in this field can be rather subjective at times. I don't know one developer at the company who was supportive of this - from what I learned, some showed up at the management office to try to convince them to take another direction. These folks were wise. What non/ex-coders should appreciate is producing high-quality software requires arming devs with next-gen tools to increase output and tighten feedback loops, without causing tool fatigue.

However, arcane tools can be super helpful as I have found recently. Ever since my wife and I had a baby in mid-2022 a side project of mine that I care for took a massive hit as I just don't have the time to work on it as much (fatherhood outweighs hobby coding of course). I was also concerned my paid work was going to take an output / quality hit. First, I decided to make a couple of changes to my setup. For Java / Kotlin programming I am continuing to use IntelliJ but have set goals to get better at using it - learning its dark corners and actually learning the productivity hacks. I have switched to using Neovim from VS Code for everything else (Go, Rust, C++, TS, Python... whatever else). The second major change that I am making is automating as much as possible while relentlessly reducing time-to-prod. A simple heuristic that I am setting myself is that if the code I commit takes longer than 20 minutes to get into production, or at very least a preview environment something is horribly wrong - either with the process that I am responsible for creating or bound to someone else's short-comings. The last major change I have made is using AI-assisted programming which has made significant advances over the last couple of years. I am most impressed by this change so far.

A fun prediction of mine for the next couple of years is this:

> Those who don't use AI assisted programming will not be as productive or grow as fast as the software developers that do.

Of course all predications carry exceptions - I don't think this will apply to those new to the field and those new to their programming careers. Afterall, how would you know if the AI wrote inefficient code? Or, how would you learn programming if a more skilled engineer wrote all the code for you? I think the same applies if an AI writes all your code for you. I'm confident in it as a tool as I have the experience to scrutinize the code in depth and challenge any security and performance downsides.

If you haven't installed the likes of Github CoPilot or tabnine, or asked ChatGPT to explain some complex code you probably should. Over 2023 I will be recommending it to the less senior and less tenured engineers I advise at work, and projects that I am responsible for delivering I will be advocating for its use. I consider it less of a code generation tool, but more of another pair of hands and eyes. I continue to believe that life rewards those who lean into change with a healthy skepticism.


Grokking tmux


`tmux` is a handy terminal multiplexer that allows you to manage multiple terminal sessions within a single window, making it a useful for tasks such as running multiple long-lived processes simultaneously, often on remote machines. It can also be used multiple development tasks easily. If you are a macOS using (like me) you can set up with the following terminal commands.

```bash
$ brew install tmux
$ tmux -V
$ man tmux
$ alias t=tmux # if you are in the 1 char command club
```

Then, opening `tmux` is as easy as running the command

```bash
$ tmux
```

Then, view the available command list with the sequence

```emacs
C-b ?
```

## Splitting panes

To split panes for easier viewing, within the current pane run the following to;

**Horizonal split**

```emacs
C-b "
```

**Vertically split**

```emacs
C-b %
```

## List sessions

To list the various active sessions run

```bash
$ tmux ls
```

## Creating Windows

Creating new `tmux` windows can be done like so

```emacs
C-b c
```

## Exiting session

Exiting a `tmux` session is as easy as

```emacs
C-b d
```

or 

```emacs
exit
```

## Attaching

To attach to a `tmux` session, run

```bash
$ tmux attach -t 0
```

## New sessions

Creating new sessions can be done with

```bash
$ tmux new -s coolsession
```

## Renaming sessions

To rename a session, you can

```bash
$ tmux rename-session -t 1 2cool
```

## Full-screen mode

To go from pane, to full-screen mobe use

```emacs
C-b z
```

## Choose session from list

To access a session from a list view you can use

```emacs
C-b s
```

## Show a clock

If having a clock in a pane seems useful, you can

```emacs
C-b t
```

## tpm (Tmux Package Manager)

`tmux` has a package manager called tpm. It can be found at [https://github.com/tmux-plugins/tpm](https://github.com/tmux-plugins/tpm). 

```bash
$ git clone https://github.com/tmux-plugins/tpm ~/.tmux/plugins/tpm
```

Once, cloned, vim into the conf file

```bash
$ vim ~/.tmux.conf
```

and list the plugins you want to install

```bash
# List of plugins
set -g @plugin 'tmux-plugins/tpm'
set -g @plugin 'tmux-plugins/tmux-sensible'

# keep at end of file
run '~/.tmux/plugins/tpm/tpm'
```

Then, refresh the current shell with

```bash
$ tmux source ~/.tmux.conf
```

## Installing the dracula theme

Installing themes is also possible. Installing the [dracula theme](https://draculatheme.com/tmux) can be done like so

```bash
set -g @plugin 'dracula/tmux'
```

```emacs
C-b I
```

## Start up with zsh

To start `tmux` on start-up of the shell you can use the following script.

```bash
if command -v tmux &> /dev/null && [ -n "$PS1" ] && [[ ! "$TERM" =~ screen ]] && [[ ! "$TERM" =~ tmux ]] && [ -z "$TMUX" ]; then
  exec tmux
fi
```

However, I do not believe this is recommended due to session nesting. It's probably a good idea not to do this.

## Awesome tmux

To conclude, further extending `tmux` and accessing reading materials can be found on this GitHub repository: [https://github.com/rothgar/awesome-tmux](https://github.com/rothgar/awesome-tmux)


Glossing TypeScript types


[General](https://www.typescriptlang.org/docs/handbook/2/types-from-types.html) and [utility](https://www.typescriptlang.org/docs/handbook/utility-types.html) types are well documented in TypeScript. I find myself writing a lot of TypeScript, regardless of what I work on, and keeping these at the forefront of my mind typically helps. Some are basic — you can't know TypeScript without knowing these. Others come into play when you find yourself working with code where the type hierarchy is a bit of a mess, and refactoring it would be difficult — usually not the task at hand. This post is nothing more than a glossary. 

## `typeof`

JavaScript already has a `typeof` operator, which can be used in an expression context. TypeScript extends this by inferring types and returning the type context to be used for a variable or property.

```ts
const person = 'Pete';
type Person = typeof Person;
```

## `keyof`

The `keyof` operator takes an object type and returns a string or numeric literal union of its keys. TypeScript automatically creates a type that matches the object's structure. The `keyof` operator gives us a union type of all the object's property names, which is super useful when you need to work with object keys.

```ts
const object = {
    name: 'Bitcoin',
    age: 16
}

type Object = typeof object;
type ObjectKeys = keyof typeof object
```

## Itersection types

Intersection types allow you to combine multiple types into a single type using the `&` operator. In this example, the `Person` type must satisfy both the `HasName` interface (having a `name` property of type `string`) and the `HasAge` interface (having an `age` property of type `number`), effectively creating a type that requires both `name` and `age` properties.

```ts
interface HasName {
    name: string;
}

interface HasAge {
    age: number;
}

type Person = HasName & HasAge;
```

## Nested types

Nested types in TypeScript allow you to define complex object structures with multiple levels of properties, mirroring the way data often exists in real applications. Using bracket notation with nested types (as shown in `NestedObject['user']['info']`) lets you extract specific nested type definitions, which is particularly useful when you need to work with just a portion of a larger type structure.

```ts
type NestedObject = {
    user: {
        info: {
            name: string;
            age: number;
        }
    }
}

type UserInfo = 
    NestedObject['user']['info'];  // { name: string; age: number; }
```

## `Record<T>`

The Record utility type creates an object type where the keys must be of a specific type (often a union of string literals) and the values must conform to another specified type. In this example, it ensures that our `Website` type must have exactly the keys `home`, `about`, and `contact`, and each of these keys must have a value that matches the `PageInfo` structure, making it perfect for mapping a fixed set of keys to a consistent value type.

```ts
type Pages = "home" | "about" | "contact";
type PageInfo = { title: string; content: string };
type Website = Record<Pages, PageInfo>;

const hello: Website = {
    home: {
        title: '...',
        content: '...'
    },
    ...
}
```

## Return types

`ReturnType<T>` is a TypeScript utility type that extracts the return type of a given function type `T`. It's helpful for reusing or enforcing consistency in types derived from function return values.

```ts
const func = () => {
    const val = 'string';
    return val;
}

type Return = 
    ReturnType<typeof func>;
```

`Awaited<T>` is yet another utility type that resolves the type of a value wrapped in a `Promise`. For example, if a function returns `Promise<string>`, Awaited extracts the inner type string. It is particularly useful for asynchronous code when combined with ReturnType.

```ts
const func = async () => {
    const val = 'string';
    return val;
}

type Return = 
    Awaited<ReturnType<typeof func>>;
```

## Function parameter types

The `Parameters` utility type extracts the parameter types of the greet function as a tuple. Since greet accepts two parameters `(name: string and age: number)`, the inferred type of `GreetParams` is `[string, number]`. This is useful for reusing or enforcing consistency with function parameter types.

```ts
function greet(name: string, age: number) {
    return `Hello ${name}, you are ${age} years old`;
}

type GreetParams = Parameters<typeof greet>;
```

### ValueOf

The custom utility type `ValueOf<T>` extracts the values of a given object type `T`. In the below example, `ValueOf<typeof colors>` accesses the `keyof T` (which are the keys of colors: 'red', 'green', and 'blue') to extract the corresponding values `('#ff0000', '#00ff00', '#0000ff')`. This is particularly useful for creating a type from the possible values of an object. I have also seen this called `PropsOf` in some codebases. 

```ts
type ValueOf<T> = T[keyof T];

const colors = {
    red: '#ff0000',
    green: '#00ff00',
    blue: '#0000ff'
} as const;

type ColorValues = 
    ValueOf<typeof colors>; // '#ff0000' | '#00ff00' | '#0000ff'
```

### `Prettify<T>`

The `Prettify<T>` custom utility type is a helpful trick in TypeScript used to "flatten" or "clean up" the display of complex or intersection types in IDEs and tools. Without Prettify, when types like NestedType are created using intersections (e.g., MainType & { isDeveloper: boolean }), the IDE might display the type as an unwieldy combination of its parts. By applying Prettify<T>, the type is reconstructed as a clean, key-value mapped object, making it easier to read and understand. It doesn't change the behavior of the type, just its appearance.

```ts
interface MainType {
    name: string;
    age: number;
}

type NestedType = MainType & {
    isDeveloper: boolean;
}

type Prettify<T> = {
    [K in keyof T]: T[K]
} & {}
```

### `Partial<T>`

The `Partial<T>` utility type creates a new type where all properties of the given type `T` are optional. In this example, `Partial<Todo>` generates a type where both title and description from the Todo interface are optional `({ title?: string; description?: string; })`. This is useful when you need to work with objects that may only have some of the properties defined, such as when creating an object incrementally, or partial editing.

```ts
interface Todo {
    title: string;
    description: string;
}

type PartialTodo = 
    Partial<Todo>;
```

### `Required<T>`

The opposite of `Partial<T>`. The `Required<T>` utility type makes all properties of the given type `T` mandatory. In this example, `Required<Config>` converts the optional properties of the Config interface into required ones, resulting in `{ name: string; timeout: number; debug: boolean; }`. Useful when you want to ensure that all properties of an object are explicitly defined.

```ts
interface Config {
    name?: string;
    timeout?: number;
    debug?: boolean;
}

type RequiredConfig = Required<Config>;
```

### `Omit<T, K>` 

The `Omit<T, K>` utility type creates a new type by removing the specified keys `K` from the given type `T`. In this example, `Omit<Todo, 'description'>` removes the description property from the Todo interface, resulting in a type with only `{ title: string; }`. This is useful when you want to exclude certain properties from an existing type to create a more specific version. If you want to omit multiple properties you pass a union type like `Omit<Todo, 'title' | 'description'>`

```ts
interface Todo {
    title: string;
    description: string;
}

type OmmitedTodo = 
    Omit<Todo, 'description'>;
```

#### `Pick<T, K>`

The `Pick<T, K>` utility type creates a new type by selecting only the specified keys `K` from the given type `T`. In this example, `Pick<User, 'id' | 'name'>` selects only the id and name properties from the `User` interface, resulting in `{ id: number; name: string; }`. This is useful when you want to create a more focused type that includes only a subset of an existing types properties.

```ts
interface User {
    id: number;
    name: string;
    email: string;
    password: string;
}

type PublicUser = 
    Pick<User, 'id' | 'name'>;
```

#### `Exclude<T, U>`

`Exclude<T, U>` creates a new type by removing all members of the union `T` that are assignable to `U`. In this example, `Exclude<Shapes, { kind: "circle" }>` removes the shape with kind: `circle` from the `Shapes` union type, leaving only `{ kind: "square", x: number; }`. This is useful when you want to filter out specific variants or members from a union type.

```ts
type Shapes =
  | { kind: 'circle', radius: number }
  | { kind: 'square', x: number };

type FromExcluded = Exclude<Shapes, { kind: "circle" }>;
```

### `Extract<T, U>`

The `Extract<T, U>` utility creates a new type by extracting only the members of type `T` that are assignable to type `U`. In this example, `Extract<Animal, 'cat' | 'dog'>` filters the Animal union type, returning only the types that are part of both `Animal` and `'cat' | 'dog'`, which results in `'cat' | 'dog'`. This is useful when you want to select a subset of types that overlap between two types or unions.

```ts
type Animal = 'cat' | 'dog' | 'bird';
type Pets = Extract<Animal, 'cat' | 'dog'>;  // 'cat' | 'dog'
```

## Conditional types

Conditional types allow you to define types that depend on a condition, using the syntax `T extends U ? X : Y`. This evaluates the type `T` against `U`, and if it extends `U`, the result is `X; otherwise, it is Y`.

In this example, `IsString<T>` is a conditional type that checks whether `T` extends `string`. If `T` is a `string`, it resolves to `true`; otherwise, it resolves to `false`. For type `A = IsString<string>`, the result is `true` because string extends `string`. For `type B = IsString<number>`, the result is `false` because `number` does not extend `string`.

Conditional types are powerful for creating flexible and type-safe logic based on type relationships.

```ts
type IsString<T> = 
    T extends string ? true : false;

type A = IsString<string>;  // true
type B = IsString<number>;  // false
```

## Mapped types

Mapped types allow you to create new types by transforming the properties of an existing type. In the example, the `Readonly<T>` mapped type iterates over all the properties of `T` and makes them `readonly`. This means that the properties of `Readonly<T>` cannot be reassigned after they are initialized.

In this case, `Readonly<Todo>` creates a new type where all properties of the `Todo` interface (title and description) become readonly, making it impossible to modify these properties after the object is created. The resulting type `ReadonlyTodo` has the same structure as `Todo`, but its properties are now immutable, i.e., readonly title: string; readonly description: string;.

Mapped types are a powerful way to dynamically create new types based on existing ones while transforming their properties, like making them readonly, optional, or adding specific transformations.

```ts
type Readonly<T> = {
    readonly [P in keyof T]: T[P];
};

interface Todo {
    title: string;
    description: string;
}

type ReadonlyTodo = 
    Readonly<Todo>;
```

## Types for state representation

Discriminated Unions are a powerful pattern for type-safe state representation, where you define a union of types that each include a common discriminant property (often called a "tag" or "status"). This discriminant allows you to safely handle different states, ensuring that you always know which variant you're dealing with.

In the example, LoadingState, SuccessState, and ErrorState each have a status property with a unique string value ('loading', 'success', or 'error'). The State type is a discriminated union of these three types. The status property acts as the discriminant, so TypeScript can narrow down the type based on its value. For example, when the status is 'loading', TypeScript knows that the type is LoadingState; when the status is 'success', it knows the type is SuccessState and so on.

This approach enables type-safe handling of different states in your application, ensuring that when you access properties like data or error, they are only available when appropriate (e.g., data is only accessible in SuccessState).

Discriminated unions are especially useful for managing different states in applications, like loading, success, and error states, providing clear and type-safe handling of these states.

```ts
type LoadingState = {
    status: 'loading';
}
type SuccessState = {
    status: 'success';
    data: string;
}
type ErrorState = {
    status: 'error';
    error: Error;
}

type State = LoadingState | SuccessState | ErrorState;
```

### Recursive Types 

Recursive Types are types that reference themselves, allowing you to represent nested or hierarchical data structures. This pattern is useful when you need to model structures that can have arbitrary levels of depth, such as trees or nested arrays.

In the example, `NestedArray<T>` is a recursive type that can be either the base type `T` or an array of `NestedArray<T>`. This allows you to create deeply nested structures, such as `number, number[], number[][]`, and so on. In the case of the example variable, it is a nested array of numbers with an arbitrary depth, specifically `[1, [2, [3]]]`, which fits the structure of NestedArray<number>.

```ts
type NestedArray<T> = T | Array<NestedArray<T>>;
// Can be: number, number[], number[][], etc.
const example: NestedArray<number> = [1, [2, [3]]];
```

### `ConstructorParameters<T>`

`ConstructorParameters<T>` is a TypeScript utility type that extracts the parameter types of a class constructor. This is useful when you want to work with or infer the parameters required to instantiate a class without needing to manually extract them.

In the example, `ConstructorParameters<typeof Person>` extracts the parameter types from the Person class's constructor, which are `name: string` and `age: number`. The resulting type `PersonConstructorParams` becomes a tuple type: `[name: string, age: number]`. This allows you to easily infer or use the constructor parameters in contexts such as function signatures, factory functions, or other type manipulations related to class instantiation.

`ConstructorParameters<T>` is helpful when you want to access or manipulate constructor argument types dynamically, making it a powerful tool when working with classes and object construction.

```ts
class Person {
    constructor(name: string, age: number) {}
}

type PersonConstructorParams = ConstructorParameters<typeof Person>;
// [name: string, age: number]
```


### Tuple types with labels

Tuple Types with Labels allow you to create tuples where each element has a named label, improving code readability and clarity by providing context to each element in the tuple.

```ts
type Point = [x: number, y: number];
type RGB = [r: number, g: number, b: number];
```

The `Point` type is a tuple where the first element is labeled `x` (representing the x-coordinate) and the second element is labeled `y` (representing the y-coordinate). While tuples in TypeScript are typically used for fixed-length arrays, adding labels to the elements helps clarify their purpose, making it easier to understand how the values are intended to be used.

This feature is especially useful in cases where tuples represent structured data (like coordinates or pairs of values), and having the labels enhances the type safety and self-documentation of your code.

