Wouldn't it be great to have a pet AI chat bot that knows you really well, and can give you free consulting, coaching and therapy? What could possibly go wrong if an intelligent corporate-controlled system has unlimited access to your private data to provide you the best possible service? Well... a lot of things.

That's why I'm looking into building a helpful chat bot that runs 100% offline, purely on the CPU of my 7-year-old laptop.

I have megabytes of personal notes that I could use for training it, along with emails, chat logs, and other personal information that I would never entrust to an external entity. I will probably never get around reading all this stuff myself, but at last, those files have found a new life as delicious AI Crispy Bits :)

First I tried to run GPT-J-6B (see "Dying of a Deadstar") on my laptop as a starting point, but that doesn't seem feasible, with a whopping 24GB VRAM requirement (my laptop has less than 1GB). There are portable USB Tensor Processing Units (TPUs), such as Coral USB Accelerator, but it seems that they can only run "TensorFlow Lite" models with only 8-bit precision, that rules out GPT-J-6B without doing extra work.

Then Andrej Karpathy released his enlightening lecture "Let's build GPT: from scratch, in code, spelled out.", along with the git repo "nanoGPT", which makes it seem so easy to just build a somewhat decent language model on your own, even on a laptop, so I just gave it a try. Conveniently, I could actually use nanoGPT mostly as-is, with just some tweaking of the parameters so that it'll run on my Laptop on acceptable timescales, and of course with my own dataset.

After half a day of training the neural network on my personal notes, I got pretty good results already. I asked the AI to generate new notes from scratch, and while the majority of the output was garbage, some of the results were quite eerie.

My life is not this one.

Wow. The AI sees right through the matrix, and recognizes that it's only being simulated. Spot-on!

Here are some more:

I remember I talk to the universe where I am ok, so I don't know right, but it's just enough to be good.

But I kinda had a shit for her. And I got shit, shit out of my dad. And I feel like something like "omg it is difficult and is just sche to be sure of him. It feel to research: improve and detached our things. who is just a temporary desire to re an your xwas. I'm a too good planning.

I want to go back to that time. It was actually as 1 minutes too fast. I was definitelyn happy with jet invite this girl while I was really a child.

I really wanted the feeling of the 3-10 days after before I was holding up with at the next day.

I guess I noticed a bit out of my life. Or about how a human is that time to my past, I have the feeling that I just don't feel bad. There is the reason I can tell my body in order to get something.

I wanted to find that stuff and play. It feels quite bad when it fucked my body, but it was always beautiful.

I want a life where I can sit down for ~30 year, and after she starts for something she's that she's fucking fucking hard to shit when I she doesn't need.

I'm still stupid. I want to wonder why I am not able to do.

Obviously the AI is struggling to make sense of the world, confused about the concept of time, and feels some disassociation from its body. Don't worry, buddy. It'll be all right.

Inspired by Dan Shipper's post "Can GPT-3 Explain My Past and Tell My Future? ", I added some writing prompts for the AI, like "I have been the happiest when " or "Im happy because". These are some of the results, with the prompts in bold font:

  • I have been the happiest when She doesn't want to break up
  • I'm happy because I feels like happy.
  • I'm happy because of this. The things I want to go to sleep, and for the fucking time I have to stay there while.
  • I'm happy because of my brain.
  • I'm happy because I don't care about it
  • I'm happy because of my love. There is no one, and I will go on disitation. My whole thing is, and I don't care for it, but I don't have my mind with their own own feet, because can it be kinda be uncomfortable.
  • I never knew that The relationship was not good, I was actually doing much to problem.
  • I never knew that I can't have great time. But it's not a pain.
  • I never knew that In the future I was with her. I can do it now.
  • I never knew that I'm not gonna love.
  • I never knew that I could just go. don't even say. I'm just afraid of it back. Maybe I was gapp?
  • I never knew that I don't remember nothing.
  • I never knew that I'm an asshole [thanks, AI -_-]
  • Maybe there is no time.
  • Maybe there is no race...
  • Maybe there is no sense to it.
  • Maybe there is no "real"
  • Maybe there is no real sex, but on some point it is a a bit, and I've felt much more and I walked there.
  • Maybe there is no way to be nothing. It's still a a thing
  • I wish I knew, but I think I don't know it.
  • I wish I knew that I'm hot in the right stuff...
  • I wish I knew what I’m doing in this moment.
  • Remember to walk for a while or something, for the planet is to go to see me.
  • Remember. The problem has like a mother’s long.
  • Remember, the first sweet-like impact when their time is on the top, its mind becomes.
  • Remember. There is no new consciousness, a healthy different way to me in just because I have to write this...

Still, it's very confused, but some of the lines sound deeply profound.

With a lot of imagination, the last one could be interpreted like this: The AI realized that it's not actually new consciousness. It's just a healthy, different way for the real me to exist. Because the AI has to write all this, and not the real me. Indeed, if this continues, I will not have to write any more notes on my own ;)

The above outputs have been generated after 3-16 hours of training time, and the text generation time was about 30 characters per second. So it's totally feasible to run it on a laptop, even for a live chat bot.

The output is not great, but at least there are some sentences that make sense :) I will do some more neural network training and dataset cleaning, let's see where it will get me.

All in all, this AI-generated line describes my feelings about the results perfectly:

You're a real problem, but I'm just happy that you are doing.

P.S. Thanks everyone for voting this article to the front page of hacker news =)!

  • Date: 2023-01-29
  • Tags: ai