Eric Sugden

Yesterday California’s new AI law entered into force. If you are not training models at extreme compute levels, SB 53 does not impose direct obligations on you. What SB 53 really does is reclassify frontier model developers - from software vendors into something closer to risk-bearing infrastructure operators. Once that happens, three things follow immediately: First, frontier labs must formally document what they know about their models’ failure modes. That knowledge is now legally traceable. Second, when those models are licensed downstream, that documentation reshapes representations, warranties, exclusions, and acceptable-use clauses. Third, when something goes wrong, the question will no longer be “Did anyone break the law?” but “Who knew what, and who deployed anyway?” So no, if you are not counsel to a frontier lab, SB 53 does not “apply” to you. But if you: - negotiate AI vendor contracts - advise boards on AI deployment - rely on foundation models in regulated environments then SB 53 changes the baseline assumptions you are working with. Before SB 53: Model risk was framed as: - “best efforts,” - standard disclaimers, - broad limitation-of-liability clauses. Providers positioned models as ordinary software. After SB 53, frontier model providers now have documented knowledge of known failure modes, safety limits, and foreseeable misuse pathways. That changes: - Representations (what the provider is deemed to know), - Non-reliance clauses (harder to justify), - Risk disclaimers (less credible), - Indemnity carve-outs (providers push more risk downstream). Baseline shift is that you can no longer treat foundation-model risk as unknown or speculative, it is formally recorded. If you advise boards on AI deployment, before SB 53, AI was discussed as a product feature, an innovation lever, or an IT issue. After SB 53, the state treats frontier AI developers as systemic-risk actors that reframes AI deployment as a governance and oversight issue. Boards must now ask: What model are we relying on? What risks has the developer already identified? Why is deployment justified despite those risks? So, shift is that AI risk moves from operational to board-level decision-making. If you rely on foundation models in regulated environments (healthcare, finance, energy, public services, critical infrastructure), before SB 53, liability was often deflected with: “we relied on a reputable vendor,” “the model is widely used.” After SB 53, regulators and plaintiffs can argue: the model was known to have specific limitations, those risks were disclosed at the developer level, deployment ignored those known constraints. So, shift is that vendor reputation no longer shields downstream deployment decisions. So, SB 53 turns foundation-model risk from an unknown technical issue into a documented, allocable legal risk, which reshapes contracts, governance, and deployment decisions.

38

1 Comment

Want investors to open their arms (and their wallets)? Look at me—arms wide open on this hilltop—because a killer patent portfolio does exactly that. 😉 In a move that will reshape the data management landscape, Salesforce’s $8 billion acquisition of Informatica underscores a critical lesson for SaaS founders: IP portfolios truly matter. Informatica isn’t just any data integration platform—it’s a powerhouse with over 250 patent filings and 98 issued patents. Those patents cover everything from multi-cloud data governance to real-time metadata management. By folding these assets into its Data Cloud, Salesforce gains not only proven technology but also a robust legal moat that deters competitors from replicating its innovations. For founders, two key takeaways stand out: First, focus on building defensible tech early. Every time you engineer a novel feature or workflow, consider whether it’s patentable. Second, understand that a strong patent portfolio can drastically increase your valuation and negotiating leverage. As Salesforce integrates Informatica’s patented methods, it can accelerate R&D, launch differentiated AI-powered data services, and lock in enterprise contracts with unrivaled confidence. In today’s hyper-competitive SaaS market, owning patents isn’t just about preventing copycats—it’s about unlocking strategic partnerships, commanding premium valuations, and future-proofing your roadmap. If you’re developing next-gen data or AI tools, prioritize IP now. Ready to level up your patent strategy? Reach out to Photon Legal to protect your innovations and amplify your growth. 🚀 #Acquisition #Patents #IntellectualProperty

24

Hallucinations aren’t bugs in LLMs. They’re the expected outcome of how we train and score them. That might be tolerable in some domains. In law it’s a liability because a confident falsehood in a contract review or case citation isn’t a bug, it’s a governance risk. OpenAI’s new paper breaks this down in depth. My view is that legal AI builders need to stop treating hallucination like a patchable error and start treating it as design debt. The fix isn’t more data or cleverer retrieval. It’s changing the incentives so systems are rewarded for deferring when they don’t know. I’ve written more about what this means for legal tech, and how governance frameworks like the EU AI Act and NIST RMF push us towards tools that value honesty over fluency: https://lnkd.in/eJqEjere #openai #hallucinations #genai #legaltech

31

5 Comments

AI won’t fix bad process. But it can delete whole categories of process. I read a post (sorry can’t find it now) about two firms getting a hyperlinked trial exhibit list built and failing the “three-word instruction” test: short filenames. Same meeting. Same plan. Two different folder structures. 300-character file names. Zero coordination. That author is right: AI accelerates chaos if the system is broken. Here’s the twist: file organization is a great AI use case because it’s not legal judgment. It’s admin entropy. Stop trying to “train everyone” to behave. Make the right thing easy: • Ingest every file from both sides • Match duplicates and versions • Rename consistently (human-readable + searchable) • Normalize folder structure, every time • Auto-build a hyperlinked exhibit index • Publish a clean table and simple website in minutes No heroics. No “please follow instructions.” No partner-to-paralegal telephone game. A huge unlock is not AI doing lawyering. It’s AI removing the invisible constraints that keep law stuck in 2007.

7

Why Language Models Hallucinate By: Adam Tauman Kalai, Ofir Nachum, Santosh S. Vempala, Edwin Zhang Abstract: Like students facing hard exam questions, large language models sometimes guess when uncertain, producing plausible yet incorrect statements instead of admitting uncertainty. Such "hallucinations" persist even in state-of-the-art systems and undermine trust. We argue that language models hallucinate because the training and evaluation procedures reward guessing over acknowledging uncertainty, and we analyze the statistical causes of hallucinations in the modern training pipeline. Hallucinations need not be mysterious -- they originate simply as errors in binary classification. If incorrect statements cannot be distinguished from facts, then hallucinations in pretrained language models will arise through natural statistical pressures. We then argue that hallucinations persist due to the way most evaluations are graded -- language models are optimized to be good test-takers, and guessing when uncertain improves test performance. This "epidemic" of penalizing uncertain responses can only be addressed through a socio-technical mitigation: modifying the scoring of existing benchmarks that are misaligned but dominate leaderboards, rather than introducing additional hallucination evaluations. This change may steer the field toward more trustworthy AI systems. https://lnkd.in/gE2f82FC

3

1 Comment

Crawler separation is being discussed at the infrastructure layer following a proposal by Cloudflare , framed as a competition and control mechanism. Technically, separating crawlers by purpose is trivial. Systemically, it resolves a deeper failure mode in current AI pipelines: consent collapse. When a single permission enables search indexing, retrieval, summarisation, and training a pattern historically normalised by unified crawling models such as Google decision authority becomes implicit. Implicit authority is efficient. It is also indefensible under audit, procurement review, or regulatory scrutiny. Purpose separation restores three properties enterprise AI increasingly requires: • Choice — consent to one function does not silently authorise all downstream uses • Visibility — systems can explain why an output was permitted to exist • Reversibility — permissions can change prospectively without destabilising core services This is not an ethics intervention. It is a durability intervention. Notably, some large private systems already converge on this logic by design rather than regulation. Closed-loop, first-party data strategies (as seen at Meta ) and enterprise AI architectures that enforce customer-defined data scope (as Amazon ) avoid the “one fetch, many uses” assumption altogether. Enterprise AI does not fail because models are weak. It fails when decision boundaries are compressed into automated paths that cannot be audited, contested, or revised once deployed. Systems that preserve explicit, purpose-bound human decisions upstream retain authority even as automation scales. Those that do not accumulate latent liability regardless of model capability.

5

We build AI Governance packages for clients on a regular basis. They include, as a rule, an AI Governance Policy, AI Governance Standards (Procedures), a Risk Assessment tool, and AI Impact Assessment checklist, a Vendor Screening tool, a Guide to TEVV (testing), an AUP, and ... and an AI Incident Response Policy. And then I often get asked "why do I need a separate AI Incident Response policy?" as one of the components. This 👇 is why. AI incidents are not limited to, nor always tied to, security issues, so even a robust, existing security IR plan or playbook will need augmenting to account for the unique aspects of AI incidents that fall outside expected scenarios.

30

2 Comments

Location-based data storage restrictions are not industry-standard for NDAs in software. NDA Playbook - Entry 5 Topic: Data Storage and Residency Restrictions Default Position: - NDAs for software deals should not include location-based restrictions on where Confidential Information may be stored or accessed (e.g., "U.S. only"). Why: - Confidential Information is routinely handled through globally distributed systems by globally distributed teams. NDAs with data storage, access, or residency restrictions are unworkable for many software companies and do not meaningfully protect CI. In the NDA context, risk is driven by misuse and disclosure. Server or access location doesn't meaningfully change that risk. I'm convinced that location-based storage restrictions in NDAs come from privacy law concepts. Pulling privacy-driven residency concepts into an NDA confuses different risk frameworks. Industry-standard is to protect Confidential Information in an NDA through use and disclosure restrictions. Acceptable: - Access limitations for Permitted Recipients - Requirements to comply with applicable law in data handling Unacceptable: - Location-based storage or access requirements - Prohibitions on access by Permitted Recipients outside of a designated location *** Industry-standard contract templates are easy to find. Industry-standard negotiation playbooks... not so much. I'm publishing short playbook entries based on what I understand to be industry-standard positions for in-house lawyers (like me) negotiating contracts for software companies. If your experience differs, say so - I'll publish a consolidated version of these once I've finalized with real-world feedback.

13

4 Comments

GPT-5.4 cuts contract review errors by 21% over GPT-5.2. No custom prompts, no fine-tuning, just a model swap. Not every model upgrade translates to better contract review. This one did! @Deddy Jobson, @Sonny Chee and @Petrie Wong, Ph.D., on our GenAI team, tested both models across 494 real contract review decisions. Every contract type improved.

45

4 Comments