AI luminary Andrej Karpathy has released a handy AI-building capacity called nanochat so that you can happily build a ChatGPT-like clone.
getty
In today’s column, I explore a newly released AI-building capability that allows you to readily devise a ChatGPT-like clone. This innovative capability is known as “nanochat” and was made available by AI luminary Andrej Karpathy on October 13, 2025. It’s exciting and getting noteworthy buzz.
Here’s the deal. A quite useful process is laid out via the nanochat capability that provides a series of specialized AI-building tools so that you can delicately undertake a do-it-yourself step-by-step procedure for creating your very own generative AI or large language model (LLM). Nice. Most of the steps are free, other than you need to have access to a server on which you can run the budding AI. Likely, the server or GPU processing time will cost you around $100 in the cloud if you aim for a low-end minimalist version.
Amidst the excitement, please be aware that there are some notable gotchas involved. Not everyone is going to have smooth sailing. If you opt to proceed with nanochat, keep your eyes wide open and consider the tips and insights that I cover in this discussion. All in all, the odds are you’ll need to have techie skills or AI prowess to make this work successfully.
Let’s talk about it.
This analysis of AI breakthroughs is part of my ongoing Forbes column coverage on the latest in AI, including identifying and explaining various impactful AI complexities (see the link here).
Ways To Build AI
Before I dive into the specifics of nanochat, I’d like to take a moment to provide some helpful context.
First, there are lots of AI-building tools that are readily available if you want to build an LLM that is akin to ChatGPT, Claude, Gemini, Grok, Llama, or anything like the major generative AIs out there. Some of those construction tools are free to use and available on an open-source basis. Others will set you back a handful of precious bucks, perhaps equal to getting a fancy haircut or eating out for dinner at a ritzy restaurant. The highly robust AI-building tools will ring up the dollars and cost you some sizable dough. As is plainly evident, the cost of the tools is a key factor.
Selecting suitable AI-building tools depends on a variety of additional factors beyond cost alone. For example, a vital question is whether you want to build a small-scale AI or a large-scale AI. Some of the building capabilities are short and sweet and end up producing a toy version of an LLM. That’s perfectly fine if you are mainly interested in learning how to craft AI. You won’t be able to use the resultant AI for anything beyond simpleton uses.
There are stronger capabilities that allow you to construct a bona fide small language model (SLM) or even a larger LLM. I’ve previously discussed that SLMs are especially useful since they are small in size and operate fully contained within a smartphone or edge device, and can act on a standalone basis that doesn’t require real-time access to the Internet (see my coverage about SLMs at the link here). The downside is that SLMs don’t usually have the breadth and depth of an LLM. You are trading off robustness for the advantage of having the AI be smaller and independent of a network connection.
The biggest kicker underlying building generative AI is the data that you’ll need to have in hand when training the nascent system. Data is king. But data doesn’t grow on trees.
You might already know that there is a lot of controversy regarding the data that is commonly used to train LLMs. The major AI makers tend to scan widely across the Internet and use all manner of writings and images to train their AI. This has raised serious and unresolved legal issues concerning whether the AI makers have violated copyright laws and essentially poached people’s online materials. For my analysis of the legal facets and debates taking place on this sobering consideration, see the link here.
I will revisit the above facets as I walk you through a 30,000-foot level perspective on building an AI system. The steps are about the same for any of the AI-building suites. I’ll emphasize nanochat, but similar steps tend to occur with other such capabilities. Also, I will henceforth, for ease of reference, use the moniker of LMs, i.e., language models, leaving out whether the LM is large or small unless there is a need to be specific.
Wanting To Build Generative AI
Suppose you are desirous of building your own generative AI or LM that can be just like the major league players, such as OpenAI’s ChatGPT.
My first question for you is what kind of AI-related background you have. If you are already versed in the techniques and technologies of AI, you can probably choose whichever of the many available AI-building kits fits your interests. A non-AI techie that isn’t up-to-speed about AI could probably do likewise, though they are going to struggle with the AI-specifics and will need to do some added research and reading to get the drift of things. A neophyte or newbie is not going to easily use an in-the-weeds tool and would be better suited to using something that hides the complexities.
The same rules of thumb apply to the newly released nanochat. An AI-versed person would enjoy using nanochat and likely admire the intricacies. A non-AI techie could use nanochat to somewhat arduously cut their teeth on learning about AI. Newbies are going to be behind the eight ball, and I would suggest they go a different route.
Karpathy has been down this road before. AI insiders might know of nanoGPT, the forerunner to nanochat. The prior nanoGPT has been around for a while and only handled the upfront steps of building an LM. The new version goes full tilt and allows you to proceed A-to-Z.
Here’s what Andrej Karpathy stated in his recent tweet:
- “Unlike my earlier similar repo nanoGPT which only covered pretraining, nanochat is a minimal, from scratch, full-stack training/inference pipeline of a simple ChatGPT clone in a single, dependency-minimal codebase. You boot up a cloud GPU box, run a single script and in as little as 4 hours later you can talk to your own LLM in a ChatGPT-like web UI.” (source: X posting on October 13, 2025).
The parlance of referring to a “repo” is short for repository. One of the most well-known repositories is GitHub, an online repository that allows software builders to store their code, data, and documentation. The nanoGPT is stored in GitHub, and so is the newly released nanochat (see the link here).
Nanochat is free to use. If you are wondering where the $100 estimated cost comes into the picture, he tried out nanochat to create a tiny LM and says that it was accomplished in about 4 hours of runtime on a server GPU charging at $24 per hour (that’s 4 x $24 = $96). That gets you are barebones LM that works pretty much at a kindergartener level. Anyone trying to earnestly scale up is possibly looking at hundreds of dollars more in run-time costs, maybe thousands of dollars, and will need a bunch more data.
Steps To Build Generative AI
Now that you’ve decided to take the plunge (yes, you have), we can briskly walk through the steps involved in developing an LM.
On a high-level basis, the effort is typically undertaken in these six major steps:
- Establish a tokenizer that converts text-based words into a numeric format.
- Train the LM with the data that you’ve obtained and suitably prepared.
- Do additional training that gets the LM to be conversational and can chat.
- Perform fine-tuning using popular techniques such as SFT and RLHF.
- Test the budding LM, identify strengths/weaknesses, and refine as needed.
- Determine whether to use “in the small” or opt to do some hefty scaling.
Let’s do a speedrun of those steps.
First, you get underway by setting up a tool that will convert text into numeric codes. The gist is that any text that is fed into the LM must be converted inside the AI into numeric values, which are generally known as tokens (for my detailed explanation of tokenization, see the link here). There are lots of tokenizers that will do this for you. Since we are trying to do everything from scratch, a place to start is to find a program that does tokenization and set it up for our endeavors.
Next, we need to find data that we aim to use for the data training of the LM. This step can be the roughest part of the project. Are you allowed to legally use the data for this purpose? Have you collected the appropriate kind of data? Do you have enough data, or will your LM fall short due to insufficient data? In addition, you almost always need to do a cleanup of the raw data before it is ready to be fed into the LM for initial training. That could be an exhausting and extensive struggle.
The bulk of the run-time for getting the LM up and going is devoted to this data training step. You feed in the data, the processor or processors whirl away, the AI algorithm pattern matches on the data, meanwhile, the system usage charges pile up. Luckily, the other steps won’t require as many processing cycles.
After the mainstay of the data training has been accomplished, the LM will be able to answer questions on a one-line basis. It won’t be able to engage in dialogues. We want our LMs to be chatty and carry on conversations. You need to spur the AI in that direction. Once the chatty cat is coming along, various fine-tuning is required. The LM at this juncture is guaranteed to periodically say foul things. The fine-tuning will give points as rewards and penalties so that mathematically and computationally, the LM will steer toward sparkling language.
Finally, you will want to test the LM and see if it makes sense to you. Additional tweaks might be necessary. The last step entails deciding whether to put more time, effort, data, and money into the venture. You might be satisfied with the simpleton version, or you might feel emboldened and decide to keep going toward a robust LM.
Some Specifics About Nanochat
I would like to share a few highlights of nanochat so that you’ll have a semblance of the particulars of Karpathy’s latest creation:
- The Script. A provided script does a lot of the nitty-gritty for you. Make sure to take a close look at the script. I suppose someone could proceed without comprehending the script, but I’d urge that you dissect the script and understand what it is doing. This is valuable as a learning experience. You might also want to make changes to the script, depending upon what resources you have available and wish to use in lieu of how the script is set up.
- The Data. I earlier noted that the data side of things is the make-or-break of building an LM. In the case of nanochat, Karpathy has chosen to use a publicly available preexisting crawl of the Internet that collected and prepared data into a dataset known as FineWeb-EDU. This is a subset of a larger dataset known as FineWeb. He then did additional preparations of the FineWeb-EDU subset and put the transformed data into a prepackaged dataset that is about 24GB in size. It is up to you to decide whether you are comfortable using the dataset, and whether it is large enough and focused on the data you want to train with.
- The Conversations. If you’ve never seen how a rudimentary LM is adapted to carry on conversations, the nanochat does a handy job of showing you the process involved. Besides chatting, another vital aspect entails training the LM on being able to answer multiple-choice questions. Believe it or not, being able to realize that a multiple-choice question requires selecting one or more of the answers from a list is not going to happen by magic alone. The LM must be explicitly trained on this feat.
- The Fine Tuning. The budding LM is going to emit expletives and unsavory responses if we don’t give it some crucial lessons in language etiquette. A popular approach to fine-tuning is known as SFT (supervised fine-tuning), and another one that further boosts the savoriness is RLHF (reinforcement learning via human feedback). Those are used during the nanochat fine-tuning process. For my overall explanation about how the techniques of SFT and RLHF work, see the link here.
Mindful Thoughts
Some mindful and concluding thoughts are warranted.
Karpathy indicates in his documentation that nanochat is an evolving project and that he plans on making various enhancements. If you look at it a month or two from now, it might be completely different from this 1.0 version. His overarching vision is this goal: “It is a single, cohesive, minimal, readable, hackable, maximally-forkable ‘strong baseline’ codebase designed to run start to end and produce a concrete ChatGPT clone and its report card.” That’s quite a mouthful.
He also stated that he intends for nanochat to be used as a capstone project in the LLM101n course that he is crafting. I’d say that’s a very fitting use of the capability. Students would have an eye-opening opportunity to step through the making of an LM. Furthermore, a savvy student could easily find ways to embellish the LM and turn the AI into a fuller SLM or LLM or shift toward a narrow LM in a specific domain, such as medicine, law, mental health, etc.
Is this AI-building capability the only game in town?
Nope.
There are plenty of other choices. The thing is, this one is under the banner of an AI superstar that gets lots of press and outstretched attention. And the tag line he conjured up is notably smarmy and wickedly catchy: “The best ChatGPT that $100 can buy.”
Take a look, kick the tires, and see if it suits you. With additional elbow grease and maybe a few billion dollars, you might just be able to produce the next ChatGPT that takes the world by storm. I wish you all the luck that tokens can buy.