Alignment

There’s something compelling about the mathematical concept of a dot product.

Taking the two words separately…

Dot - The ageless delimiter. We use it at the end of sentences. Like that one. And that one. Dots appear in our night sky - in the form of light emitting stars and light reflecting planets. Captains have used them, and the space between them, to guide their ships through dark rock filled waters. Also in the past, when we spoke in morse code, the dot marked the spaces between the long winded dash. When we started to go to places in the digital world, it was the first mark Mr. Berners-Lee thought to use describe our destinations in cyber space - info.cern.ch - and look how far we have navigated that world using those dots in the cyber sky.

Product - the output of work, something of value - “a thing or person that is the result of an action or process” - says Oxford.

A dot product is therefore this application of a historic actor in delimiting and navigation, this beautiful mathematical construct - that takes vectors - each having both a magnitude and a direction in multidimensional space, and resolve them into a single number, which represents the degree to which they point in the same direction.

I think of it as the degree to which, when a hiker came upon a fork on a cool brisk morning walk, they might perceive the fork as one route, one trail - and not two divergent ways forward through the woods.

In the world of physics, it is used to calculate the work done by a force when it displaces an object. If you apply immense force, but achieve no displacement, you’ve done no work. Said another way in an annual review - it is the result, not the effort, that matters.

One beautiful thing about the dot product is that it is a relative measure, not referenced to some absolute north star. It isn’t a judgement of who’s right or who’s wrong.

In our neural nets and artificial intelligence, the dot product plays a critical role. It is used when we do something called forward propagation - where we figure out how much an input to a neuron matches the neuron’s weight vector - for the purpose feeding forward a signal.

The process of feeding forward is multi- step. We take the output of the dot product - this beautifully simple and elegant measure of alignment, we add some bias to it, and we stick it into an activation function - a mathematical construct of our design with non-linearities of our choosing.

As a result, we get to do all kinds of things to the core measure of vector alignment.

Going back to physics - if we pick the right bias and activation function - we can do work when the object doesn’t move. Just make that zero be a 0.1 (note our little delimiting dot rears as it numerically rears its head) in your activation function - and you can start giving out A’s for effort.

Like our little synthetic neuron, we face a constant stream of ideas, concepts and proposals - and we are constantly measuring them - and their alignment to our selves.

Is this on our path, through the woods?

How we configure our activation function is everything to our wellness.

Choose an activation function that is never zero, and always positive and you will take every fork in the trail. Whether out of a sense of duty, obligation, history, legacy (all beautiful words), or guilt, anxiety, worry, fret (not so beautiful) - you can find yourself tilting at windmills, climbing hills, and fording rivers - in valleys in counties in states in countries you never planned to visit. Unaligned, off the route, orthogonal to the life you want to live - but part of the day because of your hyperactive activation function.

Similarly, you can choose a function that resolves to zero, even when there is relatively high alignment between the opportunity and your vector self. This might make for some quiet days, weeks, and months on your trail, and steady progress along the route you thoughtfully planned. But it has the potential over the longer term to rob you of important interactions, a bit to the right and left of your current path, that may be enriching and important new lands for you to explore.

Every day, our wetware is attempting to project the options presented to us to our vision of our self - applying some bias - triggering some activation criteria. We end up walking paths, committed to trail maintenance, guarding on ramps, operating toll booths, driving plows, resurfacing highways, and engineering bridges over obstacles on paths we may not event want to be on.

I am thinking today about the forks in the trail, and how aligned they are not only to my present self, but the self I want to become.

In the words of Mr. Frost, “Yet knowing how way leads on to way, I doubted if I should ever come back.”

The Road Not Taken

By Robert Frost

Two roads diverged in a yellow wood,

And sorry I could not travel both

And be one traveler, long I stood

And looked down one as far as I could

To where it bent in the undergrowth;

Then took the other, as just as fair,

And having perhaps the better claim,

Because it was grassy and wanted wear;

Though as for that the passing there

Had worn them really about the same,

And both that morning equally lay

In leaves no step had trodden black.

Oh, I kept the first for another day!

Yet knowing how way leads on to way,

I doubted if I should ever come back.

I shall be telling this with a sigh

Somewhere ages and ages hence:

Two roads diverged in a wood, and I—

I took the one less traveled by,

And that has made all the difference.