A path to green AI

Living on the edge

Those who’ve followed our writing for the past year, aside from being singularly intelligent and attractive specimens of humanity (flattery gets you everywhere ;), know that we’ve been grappling with the question of what to do about, and with, AI.

On the one hand, we use it ourselves; The new logo attached to this piece was generated by Canva’s excellent AI engine, and we have paid accounts with OpenAI, Gemini, and Lovable, all of which get regularly used for research, workflows, and tool building. The benefits in productivity and efficiency are genuinely stunning, and a necessity given the pace at which the world is changing and we need to move in order to accomplish {redacted}.

*regular disclaimer that we do not, never have, and never will use AI to generate our writing.

On the other hand, there’s the fact that hyperscalers are building data centers way faster than renewable electricity is coming online, that places like Virginia are already hammering consumers with increases in their energy costs is order to fuel more AI, and that essentially everyone agrees that similarly bad outcomes are going to happen everywhere in pursuit of a few companies being able to generate superior shareholder returns. Nuclear is not the answer, as we wrote last year and several times since.

As technologists and capitalists, we see AI as a powerful tool to create leverage and do more with less. As environmentalists, we’re frequently horrified by it and what it’s doing to our power, air, and water. This has led to a question, explored over the past few weeks and in conversation with smart friends; Is there a way to make this all make sense together, by creating a structure for, and application of AI that’s definitively and measurably Good?

We’re not talking about less-shitty AI, which is where most of the current research is focusing. Yes, models will become smaller and more efficient, compute will improve, training will optimize. These are iterative developments, not state-changes, and if anything improved performance has the potential to spike demand in a way that drives even more electricity out of the grid. When we think of Good AI, what we’re looking for is a foundation model that offers;

1. As little power demand as possible, and ideally none. If any is needed, the demand should be used to drive buildout of a surplus of renewable energy

2. Low materials usage

3. Structured for core use-cases in decarbonization. Specificity keeps the parameter count lower and simplifies training.

4. Maintains the flexibility for other usecases and applications, thus making the upside of AI applicable to broader audiences.

5. Comparable-ish performance to the best-of-breed consumer offerings from traditional providers

Somewhat surprisingly, signs point to this being almost possible. Let’s dive in. Ok, one more note; We, and a number of seriously credentialed folks, think the technological pathway described here is maybe viable in the medium & long term. That doesn’t mean it’s easy, or that all the necessary systems and tech currently exist, because they don’t. It’s a thought experiment, not a startup, although there’s at least some potential for that to change. Back to the fun stuff.

Head in the Clouds

We’ve always hated the term “cloud computing,” which implies an amorphous lack of physicality, simply not born out by the vast and tactile reality of the thing. See, for a recent example, Mark Zuckerberg threatening to build a series of data centers with a larger footprint than the island of Manhattan. The accompanying visuals call to mind a Godzilla-sized lego creation from the type of child that the other parents in school worry about growing up to be a serial killer. Which, well, if the hoodie and oversized chains fit.

For AI buildout, the definitely-not-compensating-for-anything size of Zuck’s vision actually makes sense. While we don’t plan to go super deep on the history of AI here, it’s important to at least mention the concept of a neural net; This architecture, which underpins most modern AI models, is designed to simulate the human brain by looping as many neurons (ie, processors) into understanding the weight and biases to apply to huge amounts of data. More and better neurons equals faster processing, accuracy, responsiveness, and ultimately you being able to generate company logos, recipes for breakfast burritos, support ticket responses, ways to respond to texts from that aunt, and whatever else you desire.

For many obvious reasons, it’s easier for companies in the business of AI infrastructure to have all their computational neurons placed in secured buildings with fancy cooling systems, and inside closely packed and conveniently serviceable racks. You may start to see why we think “cloud” is such a misnomer, and problematic in that the term obfuscates what’s really happening here.

But, what if it wasn’t?

The Edge

Edge computing is the term for bringing compute as close as possible to the point at which data is captured, either by way of a dedicated device or, often, via a consumer’s personal hardware. This is more common than you might think; If you’ve ever had Google Maps steer you around a recent car crash, you’ve seen some of the benefits of edge compute in action. Before anyone yells, yes, Google certainly has enormous server farms dedicated to distributing Maps notifications. Hybrid edge/cloud systems are common and standard.

Edge has real architectural advantages over cloud; It can run on hardware that consumers were going to buy anyways (ie, smartphones, tablets, computers), uses less incremental power (people are going to charge their phones daily regardless), is less sensitive to natural disasters via in-built redundancy of nodes being all over the place (knocking out nodes slows down a neural net, but doesn’t break it), and can reduce latency by being at the point of data ingestion. Edge-hosted AI, while not fully green given that devices still need to be built and charged, is much closer to it than the less-shitty approach of making massive foundation models slightly less massive.

Some of this is already happening; A top-spec Macbook Pro, which is technically consumer hardware if one can stomach the price, is capable of running an LLM of perhaps 50bn parameters (data points, roughly), albeit slowly and with significant performance compromises. This is undeniably impressive, but very few folks are going to buy a machine capable of editing big-budget Hollywood films in order to send emails and edit google docs, or even to code. You just don’t need it. Also, modern foundation models, such as those offered by Anthropic and OpenAI, start at ~175bn parameters, and can increase by another 10 times from there. *all figures approximate, given that companies don’t release exact data.

So, what to do? Our proposal is this; Create an app that allows consumers to opt into being a node in a vast neural network, capable of running trillion-parameter LLMs. Do this via a virtual-machine partition on the device, which protects data privacy and allocates a tiny portion of the device’s available compute to this activity. Yes, the devices have the horsepower. Modern smartphones are overpowered, and generally ship with ~20%+ of their capacity occupied by software and services from the chip, OS, and overall handset makers. Bumping this up to 25% would have little-to-no effect on consumer experience.

Aside from the need for a ton of compute, AI is also constrained by labeling, which is the process of seeding decision trees with initial, human-driven intelligence. What about authentication? We’ve all wasted 15 seconds when logging into an app or secured site by clicking on the pictures of fire hydrants or park benches. Swap in pictures, video snippets, and sound files of basically anything, and the process starts to look a lot like data labelling. Have each labeling problem delivered to a bunch of devices to check consensus and ensure that one person logging into something after a few beers doesn’t skew your data, and good to go.

Make the app free, and give any consumer who opts in access to build on top of the hosted foundation model, at aggressively discounted pricing. Further monetize through open MCP and allow / encourage other companies to use the material. Create a parallel universe of apps, protocols, and companies, all based on the fully clouded (is this a word? It should be a word) foundation model.

We’re then left with Green AI, at scale and with comparable (or better) performance to anything else on the market.

We said upfront that the only minor issue with this plan is that the technology to execute it doesn’t exist yet. This is true but surmountable. Let’s count down a few areas.

1. Virtual machines and data partitions are an understood technology that works… OK. It’s common enough to have multiple operating systems running on dedicated IOT devices, but at significant cost to memory and performance. The software all runs, but slowly and badly. Consumers will, of course, not opt into anything that causes even a slightly noticeable drop in their ability to scroll instagram. There’s an enormous amount of iteration needed here, but to improve performance and to cap device usage at under 5%. There are some easy optimizations here to run the nodes only at off hours, but that’s a cheat code suitable for POC stage, not scale.

2. A distributed foundation model would need, at best guess, ~100k devices to supply sufficient compute. How can we achieve sufficient performance and speed, given that connectivity at this scale is always going to be intermittent? Up the device count by 5-10x, and set rules that devices will only enter node status when connected to power and with stable wifi. Most people charge their phones nightly, and with a proper global distribution of nodes there should always be a sufficiency available.

   1. Data centers, the answer is still data centers as a backstop. Let’s simply make a rule that the data centers will be wholly and exclusively owned, that physical construction will include green materials to the fullest possible extent, and that power will be renewable only. Oh, and that the company will monitor and publish water quality reports for 20 miles in every direction, and donate heat pumps in the local community.

3. There’s no known data privacy system or framework that’s sufficiently robust to ensure that users’ data is fully protected from software running elsewhere on the device. This is the biggest obstacle by a considerable margin. Nuclear-grade encryption and running the virtual machine with an entirely different OS could potentially help, but still.

4. GTM and user acquisition will take a while and be enormously expensive.

This is a brutally tough product to build, and an expensive company to found. How can we pull it into the market? Last week, we wrote about re-establishing trust in the carbon markets, an imperfect-but-needed economic area that suffers constantly because, well, it’s really difficult to measure how much carbon is being pulled into trees, rocks, and plants with any reliability. What if we trained the model to be primed for carbon removal quantification based on user-inputted photography, and then allowed other builders to take it from there?

If every major buyer of carbon credits put in a few tens of millions in order to create a truly green AI stack, we might have something.

Wrapping up

Could this work? Technologically, maybe. We simply need to reinvent mobile data security, model architecture, connectivity, virtual machines, and a few other of the foundational technologies to modern society. We do, honestly, expect most of this to be solved in the next 2-3 years, especially given the hints that Apple, with their effectively endless resources, is at least thinking about creating a neural net architecture involving all their iphones.

This will likely not be our most popular piece. Many wonderful folks in our climate network abhor AI, and for excellent reasons. We have no argument. In an ideal world, these technologies would either not exist, or be intended for extremely limited applications in scientific research and academia. We don’t live in that world, and instead are watching AI hoover up the vast majority of early and mid stage investment capital into mega rounds. Given the amount of money already committed to data center buildouts, it’s accurate to think of the US economy as a highly leveraged bet on AI, which will continue until these companies start making money and boosting productivity, or the bubble pops. If you can’t beat them, join them, and redirect the harm into something that vaguely resembles the original tech, but is intentionally constructed to do Good.

Just food for thought, unless of course, it isn’t 🙂

Please help us scale our mission by amplifying this piece on your socials. Our writing is free and always will be. If you’re a climate founder, now or soon, and need help scaling, come find us below.

https://www.coralcarbon.io/