Episode: 4396
Title: HPR4396: AI and Sangria
Source: https://hub.hackerpublicradio.org/ccdn.php?filename=/eps/hpr4396/hpr4396.mp3
Transcribed: 2025-10-26 00:13:49

---

This is Hacker Public Radio Episode 4396 for Monday 9 June 2025.
Today's show is entitled, Why I and Sangria.
It is hosted by Operator and is about 21 minutes long.
It carries an explicit flag.
The summary is Operator talks about AI tools and travel shooting while he makes Sangria
with our U.M.
Hello and welcome to another episode of Hacker Public Radio with your host Operator.
Back by popular demand, I don't know, there's no demand.
I'm just doing two things at once and I'm bringing you on the journey.
We are making Sangria and Rumsangria and we're going to be talking about experience with
large language models and kuda and getting all that stuff to work.
Environments, environment variables and just getting stuff like kanda and kanda, min kanda,
environment variables, setup, virtualization, dockers, all that good stuff and just making
it work.
I've done various platforms of video, AI video stuff, AI audio, AI roleplay, AI speech
to text and text to speech, things like whisper, all that stuff and it's always a journey
to get it to work.
I'm going to tell you my approach at how I get stuff to run and then the level in which
it's a level of difficulty from easiest to hardest.
First thing is recognizing the setup and environment that they prefer you to be in.
If you go to a GitHub repo and you see all you see is Linux shells and nothing about
Docker or whatever, it's probably a simple script to run to get the environment set up
in Linux and guess what, if you're running Windows, good luck, we'll talk about that later,
but you want to observe what the easiest way to install the application is that they want
you to use a Docker, that they want you to use min kanda and windows, that they want
you to use, if you're in Windows, try to use the min kanda or anaconda if they have
that.
Usually it's a bad script that will download anaconda and all the mess and all the libraries
needed with it and it's a big mess, but it just works and it gives you kind of a virtual
environment to do all your stuff in and you don't necessarily have to worry about anything
except for like kuda drivers if we're doing a bit of acceleration.
So we've got some sushi, I need to put in the refrigerator, this is not having anything
to do with whatever, and I bought flowers that I need to put some water in.
So first things first, you come across a repo you want to get working.
Look at the directions, if they don't make any sense to you, look online, you know,
silly tavern installer, silly tavern tutorial or whatever, stay away from any paid services
or it's a dollar and not just signed up because they're getting referrals out, they give
you referral links or they get money for expanding whatever dumb cloud service and there is
a million of them.
There is an AI Google Docs that I have that's like all the AI services that are out there
and it's just countless, especially just for generic AI large language model stuff,
local language models services.
So one everybody is using right now is onslaught for training, onslaught on smooth, you can
run it locally or you can run it in their cloud instance with like, you know, a network
of whatever GPUs for, you know, $45 an hour or whatever, you don't need that unless you
don't have like a six gig GPU or if you're doing training you want like a 16 or 24 gig
GPU, more or less, to be safe.
So you want to, you know, if you do the video tutorial and it doesn't work, you can try
different approaches.
Number one, the first one I try is the basic, you know, run the bad script and Anaconda
does everything for you and you had this big bloated environment, but who cares?
It works, you run it, if there's changes, it doesn't matter, you can back up and zip
up that folder and say blah, blah, blah, blah, working, you know, you know, this version
of whatever it's working, now with the roses, they say, and I don't know, whatever this
is, is you cook the bottoms and put that plant food in there.
I should have it better cutting because I'm kind of just putering the bottoms, but cut
the bottoms off, that'll give me, open them up, the cooler, the air around, that's longer
they stay, you know, good looking, these are really long stems.
I have a nice shun, a nice here that I'm going to kind of chop my hand off and with
the easy, nope, there we go, nope, nope, I'm cutting the stems off, the ninja way, the
same right away, there's no working, there's not enough, I can't hold onto it, good enough,
and also the blade is probably not that sharp, I try to keep it sharp, but as soon as somebody
uses it, they put it back and then it's not sharp anymore, I just shove this into the
countertop.
Anyways, we've got our roses ready, but that's the plant food in here, so with the whole
setup of Anaconda, you don't have that option, that's not an option, whatever.
Your next step is usually Docker.
Now I don't install Docker on Windows because I don't want Docker installed on my computer,
I don't want anything installed on my Windows computer, particularly like having to hunt
down where stuff ends up on my system and is running in the background, so I don't generally
install any Windows applications on a Windows system because I'm old, and I don't want
my registry full of garbage, all right, got these roses, and so the second step is
to try to Docker route, if you want to install Docker in Windows, that's fine, I use Windows
Substance System for Linux to do everything, and essentially what you do is I have like
a Ubuntu 222 image, I say load, you know, WSL, slash, I don't know what it is, I or something
install, and then you install the Ubuntu 222.2 image or whatever, just pick an image that's
not just a Ubuntu default, because you want to reference that image for all your base
images, so you're creating a base Windows system for Linux image to use for all your other
images, and this is good because it allows you to copy, back up, all that good stuff
here, your images, and kind of maintain a clean base image, so you copy, you created this
base image with Ubuntu 222, and it's got a weird label, whatever, you set it up, you
log in, you update everything, you set it up, how you know you're going to have to have
every single system set up, usually it's an app to get update, and then maybe an app
to get upgrade with like, you know, maybe just using Ubuntu and upgrades, turn that
on with just security, and then maybe run just a security update, whatever, but app to
get upgrade, there's not really much there, want a base image, and then you have your
base image, log out, shut it down properly, make sure everything's unmounted, I haven't
had any mount issues at all, with any of the subsystem for Linux stuff, so that's one
good thing, I've had issues with VMware before, with mounting, losing mounts, and not being
able to mount, or something hanging, dangling Chad, now I have had issues with the Windows
Sessons for Linux, mounting images over the network, so if you start up Windows Sessons
for Linux, you have images on the network, it tries to pull those images to see like, what's
there, when Windows starts, and then it will hang the entire system, at least Windows 11,
so don't put your images that don't exist on a, on Windows Sessons for Linux, if it's
a temporary mount point, or a mount point that just happens to disappear if you're moving
around, it's kind of break stuff, so you'll have to remove that image while you're connected
to it, or go in there and hopefully do it on save mode or something, and somehow disconnect
it, because it was a bit of a trip trying to get everything back together, so you have your
image, base image, you've done it, you shut it down, then you do an export, so you export that
image into a single file, it should be like three gigs, very small, it's just, you know, the standard
base image, and you take that standard base image and export it, once you've exported it, you import
right back in, so you import it, but it's important to stay in an environment, have your environment
wherever you're in the current folder, so when you do the import, it will create a folder with
the name of that, Windows Sessons for Linux, and then it will dump the image in there, so I have,
I've locked create a folder, and I'll call it, you know, I'll call it master, open web UI master,
so I don't overwrite my single click stuff with jambore, so I'll be like open web UI underscore master,
then I'll say, I'll create that folder, I'll go in that folder, and then I'll say import,
open web UI master, space open web UI master, space, and then the target of the GZ
GZ file that it exports, it's not GZ if it's a tar file, and it takes a few seconds, like on
my machine, at least, to export or import that image, and once it's in courted, now you have a
base image that's called, you know, open web UI underscore master, whatever you want to call it,
we do not want power, we want sugar, there's not a ton of sugar, of course, we're gonna be low on sugar,
and that's pretty much it for the import, now, so the steps are, now you have a window system
from Linux that has whatever image name you want, it's nice and clean, now you can start to
install tutorial, whatever, now if it has Docker inside of it, you install Docker, if it's using
CUDA, then you're gonna have to install the CUDA like dev kit or CUDA kit, and there's like security
concerns about it, but you're gonna have to install the CUDA dev kit if you're doing fine tuning
or whatever, so it's a special Docker driver that goes in with Docker, so that once you mount a
Docker image, you can talk to the CUDA drivers that are in Windows basically, so before you do
all this, if you're using CUDA, I'd have three versions of CUDA installed, so all the way up to,
I don't even know what version they're on now, but I'd have three or four of them installed,
that way, if something depends on it, then you don't have to like uninstall that version and
reinstall the whatever version, and then you're running a different thing tool that uses like
122, oh great, now I have to install that one, okay, well this one uses 124, okay, well now,
so just from what I can tell, you can just keep them all installed, and you won't have too big of the
issue, I haven't had any conflicts with using the wrong thing, because the whole thing is about
Torch version, has to match the CUDA version, which has to match the, you know, toolkit,
necessarily not necessarily the toolkit version, but all that has to match, so if you had the
wrong Python and the wrong Torch or the wrong Python and the right Torch or the right Torch,
but the wrong Python or the wrong Python and the wrong drivers or the right drivers, but the
wrong Python, it's Python itself usually, drivers, Python version, your Torch version,
and your CUDA driver version, have to all kind of match and play in the same, now there's a table
somewhere, if you look for like CUDA Ubuntu table or CUDA Debian table, you'll see like the versions
of Torch that match the versions of whatever, so if it just says Torch port 16, like I don't know what
that means, if you look on this table, it'll tell you what version matches what version of Torch,
so if they don't tell you, oh, this requires, you know, Python 1.9 and, you know, this version of
CUDA, then you can just look on the Torch and kind of assume that that's the right version that matches
it, so the whole thing is a circle jerk of a mess of trying to get it all over,
so assuming you've got it all that perfect and you run their install script and their Docker
compose or you've set up the whatever, so it's running, you're good to go, now if you want to
access that Windows Assistant from Linux from outside its local host from the host machine,
so if you want to host externally or you want to host it in your house, from anywhere besides
the local host, you have to forward whatever port that that service is on using Windows Firewall,
essentially, is the easiest way to do it, and it's like NetSH, blah, blah, blah, add, and then source,
destination, bullet, and I do that for a number of things, now when you're doing externally,
you also have to forward that port too, so I could set up an external port of 1234,
the forwards to my, my computers, IP, a port of, let's say, 4444, and then I can tell that
4444, the forward all the way to port 80, or 8080, or 4443, or whatever's been used a thousand
times across my network, I can assign that over there, so if there's a conflict anywhere across the
chain, you can use port forwarding to minimize or reduce that conflict, so you want to forward
externally port 444, but you really, the service is running on 80, don't bother, who cares?
Just set up a firewall rule to forward that 4434, or port whatever, over 80, and you're good to go,
so I've done that before, switching ports up, if I'm already using a port for one service,
then I'll have the Docker container, I never change the Docker container ports, I usually set up
the port forward, the forward those ports on to the Docker container, sometimes there has been
instances where I use the NSH command, and I forward a port like to itself, and there's like the circle,
it's like a forwarding loop, and the packets go in, and they come out, and they go in, and they come
out, and they go out and they go in a circle, that's just, I don't know why that's happened before,
I think I've done like a double port forward, or something like that, where it goes in a circle,
but that's only happened like a couple of times, so our Sangrino here is two oranges,
so instead of two oranges, I'm going to do six, seven, one, two, seven,
everybody likes the oranges in there, so I'm going to do eight pain drains,
so it's pain drains, two bottles, a double, everything, everywhere, simply double,
two bottles of like a dry wine, I use the barefoot cheaper low,
and then like a cup of orange juice, and then a bunch of these tangerines, which I'm going to
freezing hands are already frozen, so I use these gorilla gloves,
they are black like work gloves, they're probably not food safe,
well, but I use them for everything, so I have food gloves,
that I don't use on outside, whatever, they usually last a whole long time,
the food ones at least, so if I take them outside, they get ruined instantly,
and these are supposed to be whatever, I should probably, these are pretty tight, these are large,
and my hands are not large vining beans, and they are tight, so if you have large hands,
you probably need to get an extra large, we have like sausage fingers, you probably need to get
extra large, because these are large, now you get old ones, they will be all stuck together,
and you can buy like foot, which is actually pretty nasty stuff, you can buy foot powder,
or pop and powder, and put it in there, and stretch them apart, and then I put them in a cookie tin,
so somebody opens up the cookie jar, and they get top one powder, which is, I think that sounds nasty,
so I'm gonna wash my hands off, but yeah, I'm gonna tear apart these, change rings,
and then add some other stuff, while I talk about the rest of next steps, so you've got your,
you know, you got your doctor set up inside a Windows system, Linux, that gives you,
once you go WSL, you have easy backup, you shut it down, I think you can just straight up copy,
most, the last things are pretty whatever, Windows, I think Windows locks,
actually Windows locks that file, so if it's mounted, you can't, whatever, you have to actually shut it down,
so you'll do a shutdown of that thing, you can do dash D terminate, and I don't know if you can do
a dash D shutdown, I think shutdown just shuts all the images down, but either way, I do a full
shutdown of all the images, and then what time is it, it's really cool to them, I do a shutdown of
all the images, just to make sure I'm not doing the wrong thing, and then I zip them up, I zip up the,
the VS mother VHDX file, it takes up quite some time, what you can do is for files that,
you know, you don't need to compress, or we're gonna take a long time, you can actually mount those
on the native drive, so what I need, what I should do is have Olamas files dump on the root,
the parent system, the C drive, have those mounts be stored, so you can tell Olama where to store
the Olims, so what I should do is have my backup folder and have a folder that's called
like that Olama, and all my dot Olama stuff, all my Olama parameters and all my config stuff
goes in there and I can easily back that up, without having like, you know, three backups of
you know, Olama images that are like, you know, 20 gigs or something, each, I think my whole setup is
like 90 gigs, for all the Olamas stuff I have now, oh I did get eggs, look at that man, I completely
forgot, I forgot eggs when I went to the store, and I was like huh, I need to get eggs, and I remember
when I was at the store, I went to the store for other reasons, so that's why I get my ads,
my 80d meds, so I wouldn't forget to get my 80d meds, so I could remember to get my 80d meds,
so now I gotta go back, and remember to go back to pick up my prescription that I have to a little
drop, it's the whole thing, why is this keep going off, making noises, oh look, the prescription's
ready, I just got home like 10 minutes ago, I think I have two or 130, so unfortunately you guys are
screwed, so we have our either, you know, kind of environmentalized, what you want is environmentally
friendly, whatever, so either use the conda environment and you can easily back that up,
if it's mini conda, regular conda, whatever, you can back up that folder and more or less be sure,
you can keep the folder path structure, a lot of times they don't, they don't make it completely
portable, so you gotta keep the folder paths, the note of the folder paths, the structure of the
where the files are, how it moving around, I don't trust any of these guys to do it right,
now if that doesn't work again, the next step for me is I use jamboree, because jamboree creates
a portable python, node, yet, if you want to do postgres, so it looks like you got caught up after 20
minutes, really, I was just wrapping it up, saying the longer you let this angry, I said the better,
I don't put a whole lot of, I put line juice in it, no actual line, and longer you let it sit the
better it tastes, you get the little fruit of the bottom, try to get people to fruit and use the
little that has the spout, that's pretty much it, I pretty much talked about the, the
troubleshooting stuff, but I just want to say if you're here in this recording episode,
or wake up, because I'm rambling at this point, big and easy, then,
you have been listening to Hecker Public Radio, at Hecker Public Radio, does work,
today's show was contributed by a HBR listener like yourself, if you ever thought of recording,
or cast, and click on our contribute link to find out how easy it really is, hosting for HBR has
been kindly provided by an honesthost.com, the internet archive, and our sings.net,
on this otherwise status, today's show is released on our Creative Commons,
Attribution 4.0 International License.