Trustworthy vs untrustworthy software

Ken Thompson, upon receiving the Turing award, wrote a terrifying paper. “Reflections on Trusting Trust” illustrates a scenario of original sin. Because the C compiler is written in C itself, a compromised compiler can self-replicate with no trace in source code. — If you can’t trust your compiler, you can’t trust any compiler you build with it either. Sin in the family tree, no matter how distant, can propagate to your clean code even decades later. — This creates a recursive paradox. What if the tools used to validate a compiler, are themselves products of said compiler? Source review never guarantees security. You have to verify the entire ancestry. — In critical applications, a special technique known as “Diverse double-compiling” reduces risk. Two independent compiler lineages are unlikely to contain identical backdoors. Therefore, comparing the resulting binaries offers *some* level of trust.

The Hidden Risks of Pirated Software  When it comes to managing your organisation's software needs, the allure of pirated programs can be tempting. They promise the same functionality without the cost, but this shortcut comes with a hefty price that you can’t afford to overlook.  👀 Compromising More Than Costs  Using pirated software might seem like a quick fix to budget constraints, but it significantly undermines your cybersecurity. Unlike legitimate software that comes with security assurances, pirated versions often harbour malicious code. Once installed, this software can become a gateway for cybercriminals to access your sensitive organisational data. Imagine unknowingly inviting a thief into your home; that’s what pirated software does to your network.  👀 Legal Repercussions and Reputation  Beyond the immediate security risks, the legal implications of using unlicensed software can tarnish your organisation's reputation permanently. In today's digital age, maintaining trust is paramount. For SMEs and nonprofits, where resources are scarce and public perception is vital, being caught with pirated software can lead to legal battles, hefty fines, and a damaged reputation that could dissuade donors and partners.  👀 Operational Instability  Pirated software often lacks essential updates and support available to legitimate users. This can lead to software malfunctions and system failures that disrupt your operations. For an organisation striving to operate efficiently, relying on unreliable tools means risking your productivity and operational continuity.  👀 Investing in Legitimacy  The good news is, that investing in legitimate software is more feasible than you might think. Many software companies offer special licensing options for SMEs and nonprofits, providing affordable access to tools that come with full technical support and upgrades. This not only enhances your operational capacity but also fortifies your cybersecurity measures.  👀 Securing Your Digital Assets  As a leader in your organisation, it’s your responsibility to ensure every aspect of your operation is secure, legal, and efficient. Embracing legitimate software solutions not only protects your systems from cyber threats but also positions your organisation as trustworthy and reliable. It’s about making a strategic investment in your security, compliance, and overall success.  Choosing to use legitimate software is not just an operational decision; it's a commitment to upholding the integrity and sustainability of your organisation. Remember, in the fight against cyber threats, cutting corners with pirated software is a risk you cannot afford.  Help us spread our message by sharing this post with your network.  #CEO #SME #Nonprofit 

🌳 Design Patterns For Building Trust. With practical guidelines for designers on how to make products — AI and non-AI — more trustworthy, reliable and honest. In the noisy and polluted world today, trust doesn’t come for free. It doesn’t emerge by default. It must be earned and meticulously preserved — by being reliable, accountable and treating customers with respect. This holds true for people but it also for software. According to Anyi Sun, there are 5 psychological foundations of user trust: 1. Reliability 🔰 The degree to which the product consistently behaves as expected. It's a sense that that the product is dependable — based on a track record of past actions. Reliability comes from promising what you do, and doing what you promised. 2. Technical competence ⚡ Perceived intelligence, sophistication and capability of the product. It's user's belief that the product can successfully perform what they are being trusted to do. It's about trusting product's capability. 3. Understandability 🧠 The extent to which users feel they can understand how the system works or why it made a certain decision. The product must be able to articulate how a decision came along, with references to fragments that underpin a decision. 4. Faith and Care 🌱 Emotional, almost "blind trust" in the product, especially when users don't understand the underlying logic. It's a belief that the trusted party actually cares about the positive outcome for you, and intends to do good. 5. Personal attachment 🌳 A sense of rapport, connection or emotional engagement with the product. Typically it emerges when a user feels that they get meaningful value from the product, and from interactions with people supporting it. Personally, I would also add the value of repeated positive experiences that build confidence in the quality of the product, and hence its reliability. --- With AI products, hitting all these psychological foundations is extremely hard. Surely some people trust AI almost instinctively, others are more critical. But people's attitude often changes dramatically once they realized that they've made severe mistakes because of AI. Recovering from it is very hard. We can help with some design patterns: 1. Avoid "Ask me anything" → push for scoping and constraints 2. Slow down users in prompting → request specific details 3. Present multiple viewpoints, explain that experts disagree 4. Allow users to manage “memory”, profiles personalization 5. Highlight what is AI-generated and what isn't (AI disclosure) 6. Allow users to override AI-generated suggestions manually 7. Allow users to tweak AI output and refine it for their needs 8. Adapt AI's tone depending on the severity of user's task Trust is why people stay or leave. It builds long-term loyalty and helps users overcome hesitation. But it must be designed and retained — across all psychological foundations and with thoughtful UX work. I think designers will be quite busy for years to come. #ux #design

AI without trust is like a supercar without brakes. Powerful but dangerous. Originally posted on Trust3 AI Consider this split: Without Trust Layer: → Black box decisions → Unknown biases → Hidden agendas → Unchecked power With Trust Layer: → Transparent processes → Verified outcomes → Ethical guardrails → Human oversight The difference matters because: - AI touches everything - Decisions affect millions - Stakes keep rising - Trust determines adoption What we need: → Clear audit trails → Explainable outputs → Value alignment → Democratic control Remember: Power without accountability? That's not innovation. That's danger. The future needs both: → AI advancement → Trust infrastructure Which side are you building for?

⚠️ 𝐒𝐥𝐨𝐩𝐬𝐪𝐮𝐚𝐭𝐭𝐢𝐧𝐠: 𝐀𝐈 𝐇𝐚𝐥𝐥𝐮𝐜𝐢𝐧𝐚𝐭𝐢𝐨𝐧𝐬 𝐅𝐮𝐞𝐥 𝐚 𝐍𝐞𝐰 𝐒𝐮𝐩𝐩𝐥𝐲 𝐂𝐡𝐚𝐢𝐧 𝐓𝐡𝐫𝐞𝐚𝐭 ⚠️⁣ ⁣ 𝐒𝐥𝐨𝐩𝐬𝐪𝐮𝐚𝐭𝐭𝐢𝐧𝐠 is a new software supply chain attack that exploits AI hallucinations. AI coding tools like 𝐂𝐡𝐚𝐭𝐆𝐏𝐓 𝐨𝐫 𝐆𝐢𝐭𝐇𝐮𝐛 𝐂𝐨𝐩𝐢𝐥𝐨𝐭 sometimes fabricate package or library names that don’t actually exist. Attackers are now squatting on these AI-invented names by 𝐫𝐞𝐠𝐢𝐬𝐭𝐞𝐫𝐢𝐧𝐠 𝐭𝐡𝐞𝐦 𝐚𝐬 𝐫𝐞𝐚𝐥 𝐩𝐚𝐜𝐤𝐚𝐠𝐞𝐬 – 𝐛𝐮𝐭 𝐩𝐚𝐜𝐤𝐢𝐧𝐠 𝐭𝐡𝐞𝐦 𝐰𝐢𝐭𝐡 𝐦𝐚𝐥𝐰𝐚𝐫𝐞. It's like typosquatting – 𝘵𝘩𝘦 "𝘵𝘺𝘱𝘰" 𝘪𝘴 𝘣𝘺 𝘈𝘐, 𝘯𝘰𝘵 𝘢 𝘩𝘶𝘮𝘢𝘯.⁣ ⁣ When a developer blindly uses code from an AI that references one of these fictitious libraries, they could inadvertently download the attacker’s malicious package, leading to a compromise. ⁣ ⁣ 💡 𝐏𝐞𝐫𝐯𝐚𝐬𝐢𝐯𝐞𝐧𝐞𝐬𝐬 & 𝐒𝐜𝐚𝐥𝐞: Researchers found roughly 20% of recommended dependencies from popular AI code assistants were non-existent hallucinations, yielding over 𝟐𝟎𝟓,𝟎𝟎𝟎 𝐟𝐚𝐤𝐞 𝐩𝐚𝐜𝐤𝐚𝐠𝐞 𝐧𝐚𝐦𝐞𝐬 in their study. Worryingly, these hallucinations were often persistent (reappearing across multiple runs) and believable (names often resembled real packages). ⁣ ⁣ 🔺 𝐖𝐡𝐲 𝐢𝐭 𝐌𝐚𝐭𝐭𝐞𝐫𝐬: It can poison your software supply chain – a malicious dependency can compromise your entire application or infect downstream systems. This risk is amplified by developers’ over-trust in AI-generated code. One researcher proved it: they published a fake package name that an AI invented, and it was downloaded over 32,000 times. If that package had been malicious, imagine the fallout. ⁣ ⁣ 🛡️ 𝐌𝐢𝐭𝐢𝐠𝐚𝐭𝐢𝐨𝐧 – 𝐒𝐭𝐚𝐲𝐢𝐧𝐠 𝐒𝐚𝐟𝐞:⁣ 🛡️ 𝐃𝐨𝐧’𝐭 𝐭𝐫𝐮𝐬𝐭, 𝐯𝐞𝐫𝐢𝐟𝐲: Treat AI suggestions as helpful hints, not gospel. Double-check any package or link AI suggests – verify on official repositories that it actually exists and is trustworthy.⁣ 🛡️ 𝐈𝐦𝐩𝐥𝐞𝐦𝐞𝐧𝐭 𝐜𝐨𝐝𝐞 𝐫𝐞𝐯𝐢𝐞𝐰 & 𝐜𝐨𝐧𝐭𝐫𝐨𝐥𝐬: Require manual review/approval for any new dependency introduced via AI-generated code. Don’t add unknown libraries without due diligence.⁣ 🛡️ 𝐔𝐬𝐞 𝐬𝐞𝐜𝐮𝐫𝐢𝐭𝐲 𝐭𝐨𝐨𝐥𝐢𝐧𝐠: Use dependency scanning tools to catch malicious packages.⁣ 🛡️ 𝐄𝐝𝐮𝐜𝐚𝐭𝐞 𝐚𝐧𝐝 𝐚𝐝𝐚𝐩𝐭: 👨💻 𝐓𝐫𝐚𝐢𝐧 𝐝𝐞𝐯𝐬: Slopsquatting + LLM hallucinations are real threats. 🧠 𝐌𝐢𝐧𝐝𝐬𝐞𝐭: Trust AI… but always verify. 🤖 𝐏𝐫𝐨 𝐭𝐢𝐩: Ask the AI if a package is real- it’ll admit it made it up. 𝐁𝐨𝐭𝐭𝐨𝐦 𝐋𝐢𝐧𝐞: AI coding assistants boost productivity, but integrity is key. Slopsquatting shows a single hallucinated dependency can open the door to malware.🔒💡 ⁣ ⁣ 𝘍𝘰𝘳 𝘔𝘰𝘳𝘦 𝘐𝘯𝘧𝘰:⁣ csoonline.com⁣ securityweek.com⁣ ⁣ 𝐈𝐥𝐥𝐮𝐬𝐭𝐫𝐚𝐭𝐢𝐨𝐧 𝐀𝐭𝐭𝐚𝐜𝐡𝐞𝐝: 🚨 Attack Flow: Bad actor uploads fake, AI-hallucinated package (Pkg X). Dev blindly trusts LLM, installs it. 💥 Malicious code gets in. ISACA New York Metropolitan Chapter Tim Wei Teena Eugene Christina Alyssa

🚘🔐 AUTOSAR OS: Trusted vs Non-Trusted Functions — Why They Matter in ASIL ECUs In safety ECUs, one faulty SWC must NOT crash the whole system. That’s why AUTOSAR OS introduces Trusted and Non-Trusted functions. Let’s break it down 👇 ⸻ 🧠 1️⃣ The Core Idea AUTOSAR OS (OSEK-based) supports: • 🔒 Memory Protection (MPU) • ⏱ Timing Protection • 🚧 Service Protection Applications run in User Mode (non-privileged). Kernel/Trusted code runs in Supervisor Mode (privileged). But what if an application needs access to protected hardware or memory? 👉 It cannot directly. 👉 It must use a Trusted Function. ⸻ 🔍 2️⃣ Non-Trusted Functions These run in User Mode: • ❌ No direct hardware register access • ❌ No kernel memory access • ✅ Protected by MPU • ✅ Fault isolated per application 🛠 Use Case: Sensor processing, control algorithms, diagnostics logic. If it crashes → OS terminates only that application. System remains safe. ⸻ 🔐 3️⃣ Trusted Functions Configured in OS and invoked via: CallTrustedFunction(FunctionID, &params); Flow: User Task → CallTrustedFunction() → OS switches to Supervisor → Execute → Return to User Mode Capabilities: • ✅ Access restricted memory • ✅ Modify privileged registers • ✅ Execute critical low-level operations ⚠ Only minimal code should be trusted. ⸻ 🚀 4️⃣ Real Automotive Use Cases 🧯 Watchdog Access Wdg registers are privileged. Application triggers watchdog via trusted gateway. 🔋 Flash Programming Bootloader unlocks, erases, and programs flash through trusted function. 🛑 Safety Relay / High-Side Driver Control Direct hardware access must be controlled. 🧠 Protected Safety RAM Updates Safety counters or freeze frame storage in protected memory. ⸻ 📊 Trusted vs Non-Trusted Feature Non-Trusted Trusted Mode User Supervisor MPU Protection Yes Bypassed Hardware Access Limited Full Risk Low High Code Type App logic Critical low-level ⸻ 🧩 5️⃣ ISO 26262 Perspective Functional safety requires: • ✔ Freedom from interference • ✔ Spatial isolation • ✔ Fault containment Trusted functions provide controlled privilege escalation, not unrestricted access. Without this separation? A single bug could corrupt OS kernel memory → ASIL-D failure. ⸻ 🎯 6️⃣ Best Practices ✔ Keep trusted code very small ✔ No loops or blocking calls ✔ Validate all parameters ✔ No dynamic memory ✔ Perform atomic operation & return Think of trusted functions as: A surgical tool — not a workspace. ⸻ In ASIL-C/D systems, mastering this mechanism is critical for building robust, fault-tolerant ECUs.

As an industry, we’ve poured billions into #ZeroTrust for users, devices, and networks. But when it comes to software - the thing powering every modern business, we’ve made one glaring exception: OPEN SOURCE SOFTWARE! Every day, enterprises ingest unvetted, unauthenticated code from strangers on the internet. No questions asked. No provenance checked. No validation enforced. We assume OSS is safe because everyone uses it. But last week’s #npm attacks should be a wake-up call. That’s not Zero Trust. That’s blind trust. If 80% of your codebase is open source, it’s time to extend Zero Trust to the software supply chain. That means: • Pin every dependency. • Delay adoption of brand-new versions. • Pull trusted versions of OSS libraries where available. #Google's Assured OSS offering is a good one for this. • Assess health and risk of malicious behavior before you approve a package. • Don’t just scan for CVEs—ask if the code is actually exploitable. Use tools that give you evidence and control, not just noise. I wrote more about this in the blog linked 👇 You can’t have a Zero Trust architecture while implicitly trusting 80% of your code. It's time to close the gap and mandate Zero Trust for OSS. #OSS #npmattacks #softwaresupplychainsecurity