Where's the Software Supply-Chain (pt.1)

Software engineering has a problem. Let us try to use an analogy. Software engineering is like manufacturing. Nope. There's plenty said about software engineering not even being a thing - The Atlantic says so. I don't necessarily agree with that premise - I think there is a difference between a developer/programmer and a software engineer - the former are functions/attributes of the latter. An engineer, indeed, must have the full scope of the product/service in mind to include requirements, procurement/contracts/acquisition, compliance, delivery and sustainment.

I want dig in to that analogy just a second. I think we can all agree that a pencil is an "engineered" thing. And this it has a formal manufacturing process that supports the delivery of the engineered product. So...

[How a pencil is made]

Consider a simple supply-chain for the manufacturing of a simple #2 pencil. A pencil has four components: the wood shaft, the lead, the eraser, and the ferrule (the metal ring holding the eraser to the shaft). Wood for #2 pencils is forested mostly in the Sierra Nevadas of northern California. Lead is a mixture of graphite and clay. Graphite is a limited natural resource only found in a few places around the world, where as clay can be mined from about anywhere where water consistently collects (like lakes and river bottoms). The eraser is a petroleum based rubber product - many countries have localized petroleum resources. And the ferrule is an aluminum-based product. Over 1/3 of the world's aluminum sources are found in the country/continent of Australia. A #2 pencil manufacturer will source resources based on cost, availability and needs - dealing with multiple providers to prevent vendor-lock and sole-sourcing. A contract exists between the manufacturer and each provider, defining specifications for goods and deliveries. When the goods are shipped, they are tracked to ensure the resources are lost or tampered with. Upon delivery manufacturer personnel receive them and verify tolerances before distributing them internal for processing, still tracking each lot appropriately. The actual manufacturing process is well defined, calibrated and tested routinely to ensure precise tolerances for quality are met. As each pencil is produced it is tracked in a similar chain as it is packaged for downstream delivery - to a contract-based consumer/vendor. There are a lot of relationships in this chain, as well as many gates where controls exist to manage the risk and quality of the manufactured goods. The logistics of materials are controlled, sources are vetted, and engineering specifications are tightly managed for changes. In order to manage the supply-chain, data for all resources is processed and the information is available for business automation and monitoring.

There's a lot to discuss here. But I want to highlight and noodle on a couple of key points - and the relationship to software engineering.

First (and the context for this pt.1 post) is the notion of raw, upstream, resources. Let's put that in to context for software. All software is built on and by other software - the idea of raw resources is a stretch, but if we apply some liberty to the term I think we can all agree that software such as languages themselves and their build and packaging mechanisms can count. The same can be said for a language ecosystem's downstream library and framework layer. In the case of a pencil, there is a formal relationship between a resource provider and the manufacturer (mapped to software's developer/development organization) that probably exists in contractual form. And there probably is some form of tolerance, warranty, schedule, etc. that is applied in that relationship. In software we skirt around these things. While in some "types" of software - those that are "mission" or safety-critical there are indeed depths of contractual and usage relationships. But what about the rest of it, the non-safety-critical wares, the open source? There is a challenge here. But let's keep digging. In pencil manufacturing there is likely an individual (possibly not an engineer) who is responsible for the contractual relationship - but is working against a very well-defined specification (that defines exactly what is needed, when and what tolerances are allowed). And there is likely another person responsible for "receiving" the raw resources, verifying them against the contractual expectations, and making them available for the manufacturing process to use. In software development, a developer determines a need for a "raw" resource, does whatever diligence they determine adequate, and now that software and all downstream consumers are dependent on that decision. In some cases there may be formal review of such decisions - but I would assert that's not the norm.

What exactly does diligence mean here, for the developer(s) making a decision? I can tell you with a level of certainty that most developers will search for a solution, ask questions in forums/communications channels/commercial providers, and provide a cursory glance at the source if it is available (if it isn't, well hopefully they move on to the next option). Not bad. But definitely not the same as a formal engineering process - with the rigidity of contractual and tracking obligations of say a pencil. Do developers continue to manage that relationship, continuously evaluating upstream resources for better options, or even to stay current with newly available software? So why is this? I'll dive into this challenge in the next post, Pt.2.